LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: spalluru

Hi ,

Got couple of questions

We have got Hadoop as data source for most of our projects and on Hadoop we are implementing security authentication based on kerberos certificates

Wanted to know whether LAE client/BRE and lavastorm server support kerberos based authentication ?
Please note that we are running LAE on Linux server

Appreciate your quick response on the same
-Sreekanth

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: awilliams1024

Hi Sreekanth,

at this time we do not support Kerberos authentication. However, this was already marked as an item on our roadmap.

Are you intending to user both JDBC and ODBC or is one your prefered method?

Regards,

Adrian

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: spalluru

Hi ,
We are presently planning use JDBC drivers.
May I know do you have any documentation on installing ODBC drivers for Oracle , Hadoop on LAE 4.6 server running on Linux ?

-Sreekath

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: awilliams1024

Hi Sreekanth,

the ODBC driver installation and configuration information has now been sent to you by email.

Adrian

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: Tris_717684

Hello Adrian,

Do you have any driver available for JDBC so I can connect LAE to Hadoop? If not, can I please request some guidance on how to resolve the error "unable to load db driver: cloudera.impala.jdbc4.Driver"?

Thanks in advance for your kind help!

Regards,

tris

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: awilliams1024

Hi tris,

Lavastorm do not redistribute JDBC drivers for Hadoop platforms. You will need to obtain the drivers from the appropriate Hadoop distribution vendor's website, e.g.:

Cloudera - https://www.cloudera.com/downloads.html

Hortonworks - https://hortonworks.com/downloads/

** Upfront caveat: There are many issues that can prevent a successful JDBC connection to a Kerberos-secured Hadoop cluster. I do not have a secure cluster available to test this against so your mileage may vary on this ...

The driver you are using has to be compatible with the Java version used by Lavastorm. Lavastorm 6.1 uses Java 7. Check the system requirements in the driver's installation guide.

I have not used Impala but I have seen some issues with loading other JDBC drivers for Hadoop (specifically Hive) and the underlying cause may be the same. The driver typically comprise a number of jar files and there may be issues conflicts with the core jars used by Lavastorm if the driver jars have been copied into the lib/java/ext directory.

This can be overcome by:
a) Extract the driver files and copying them into an accessible directory on the machine hosting Lavastorm.

b) Declare a text type property on the JDBC Query node named 'Classpaths' and put it in the 'Java' property group.

c) Enter the file path to each of the driver's jars in the 'Classpaths' text property. Use a separate line for each path.
If required you could use a graph run property to specify the directory portion of the file paths so that you can change
this more easily if you want to run the data flow on different machines.

An example of creating a node with the custom Classpaths is shown below.

Hive_JDBC_Query_Node_with_Custom_Classpaths_Example.brg

In my case the 'HiveDriverFP' property was set to the directory where the jars had been copied to e.g.
C:/Program Files/Lavastorm/LAE6.1/ExternalDrivers/Cloudera_HiveJDBC41_2.5.16

Check the driver's installation and configuration guide for the appropriate information required to configure the driver. The node in the example reflects connecting to a cluster that is using username authentication. You will need to change the configuration if you are trying to use Kerberos, and it will have to align with the method by which the driver obtains the TicketGrantingTicket and Kerberos Principal.

In particular you will need to modify the connection string in the DbUrl property. e.g. (yours will probably be different depending on your driver version and Kerberos set-up, and may need more or fewer attributes)

jdbc:hive2://hiveHostname:10000;AuthMech=1;KrbRealm=HIVEHOSTNAM E.LAVASTORM.COM;KrbHostFQDN=hiveHostname.lavastorm .com;KrbServiceName=hive

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: Tris_717684

Thanks so much for all your inputs Adrian!

One major thing I observed with the graph you provided is my JDBC node doesn't show the "Classpaths" tab; I've got all tabs showing except this.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: Tris_717684

I figured it out Adrian!

I will reach out when I face another hurdle... meanwhile, I shall work with our engineers.

Happy Fri-yay!

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: Tris_717684

Hello Adrian,

Can I please have the documents you shared with Sreekanth? I had been successful with connecting LAE to Hadoop previously. However, with the my OS upgraded to Windows 10, I lost this connectivity. Perhaps these documents may be able to assist with our troubleshooting/fixing the problem.

Many thanks!

Regards,

tris

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: awilliams1024

The Hive ODBC Driver configuration guide is shipped with the LAE 6.1 installation software and should be in the following location (assuming the default installation directory):
C:\Program Files\Lavastorm\LAE6.1\docs\lal

It is also attached here for completeness:

Lavastorm ODBC Driver for Hive Configuration Guide.pdf

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: Tris_717684

Many thanks Adrian! I shall work with our Hadoop Engineer.

Regards,

tris

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: Tris_717684

Hello Adrian,

Despite numerous efforts to get my Hadoop node working, the engineer remains unsuccessful.

What the company did was to revert my computer to Windows 8.1. Thankfully, the same Hadoop node now works, as it was previously. In saying this, I think we have isolated the problem and it's relating to the Windows 10 OS. I dont know if this is a known problem within the LAE community, but I thought I'd share nonetheless.

I'm unsuccessful in attaching the screenshot I took but the error showing is: "ERROR: unable to connect: [Cloudera][Hive JDBC Driver](500168) Error creating login context using ticket cache: Unable to obtain Principal Name for authentication."

Given this, do you know if there are known solutions for LAE to work in Windows 10?

Many thanks in advance!

Regards,

tris

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: awilliams1024

I have not heard of any specific issues relating to operation on Windows 10 but I will check with my colleagues.

In the meantime:

I saw this post that is reporting the same error.
https://community.cloudera.com/t5/Ba...iver/m-p/60227
The reporter stated that the approach specified in their second comment worked in one situation but did not work for RStudio.
Maybe the successful approach may provide a clue.

Also, this webpage provides some information on troubleshooting the same error at the bottom of the page:
https://justnumbersandthings.com/pos...-dbeaver-hive/

Also, which version of LAE are you using?

Thanks