Data360 Analyze

I am new to LavaStorm and am working with a Teradata system with extremely large datasets.  A nice method of pulling data down with Teradata is FastExport.

Does anyone know how to get this to work with Lavastorm?  I've seen comments mentioning Python but I haven't used Python before to get this to work and would need help in doing so.

Thanks!!

Dear Jon,

It sounds like you can use Teradata's FastExport feature over JDBC.  Have you tried this approach?  You can install the Teradata JDBC driver by placing it (and any related JAR's) in the following directory:

C:\Users\<your user name>\AppData\Local\Dataverse\site\lib\java\ext

Once those JAR's are copied in, you should be able to use the Teradata FastExport feature immediately.  No reboot required!  Be sure to put "TYPE=FASTEXPORT" into your connection string in order to turn the feature on!

Feel free to write back if you need any more help getting this driver working.

Rocco

Thanks for the reply!  I'm working to get the drivers to do this.  

But I have a question.  Have you ever used TPT with CLI and Lavastorm?  

Dear Jon,

I am unfamiliar with the acronym TPT.  I presume that that is a Teradata product.   Could you please provide me with more detail?

Thank you,

Rocco

Rocco,

Here is what it is:

http://www.teradata.com/tools-and-utilities/parallel-transporter/

Yes it is a Teradata Product.  But some tools can utilize it so I'm wondering if Lavastorm can with jdbc.  Probably with a Python script but I'm unfamiliar with Python at the moment.

Dear Jon,

I think that I may have misread your question.

When you say "TPT with CLI", are you talking about the command-line version of TPT or about using TPT over Teradata's Call-Level Interface?  I have found evidence that TPT can be run from a prompt, but I want to make sure that I understand your question before I provide an answer.

Thank you,

Rocco

Yes, that is what I'm referring to.  It can be run from a prompt which I'm currently working on building a script on my computer that I can then transfer to Lavastorm and kick off from there.  

Are you familiar with using a variable from a previous node to query Teradata?  I'm trying to do this, I had it working with ODBC but am now trying CLI and am getting this error when I put the below in.  'Bind Name in Teradata SQL Statements must not begin with a digit.'

SELECT *

FROM TESTTABLE

WHERE Test = :1

If I switch it to :? or :Test I still get errors (but different ones).

Any thoughts?

Which node are you using to do the query? DbQuery doesn't support variables.  You'd have to use DbExecute.

query="SELECT * FROM TESTTABLE WHERE Test = :?"
sqlSelect(1,query,myField)

in this case,
"Test" must be the name of a column in the database
"myField" must be a name of an input column

The two names can be identical. I am just demonstrating that they can be different.

I have seen a few implementations where this will work;

query="SELECT * FROM TESTTABLE WHERE Test = :myField"
sqlSelect(1,query,myField)

but you still have to name the input column as a "Bind Column" on the sqlSelect statement.

===
If you are using JDBC nodes (any of them), you must put the input column name into the parameter "SqlQueryFieldBindings"  and then  you  can use :? or :myField  in the query

Dear Jon,

Since you prefer using TPT via the command line, I suggest running your command line script by hand along with your Dataverse data flow. In spring, some additional automation features will become available in the desktop product along with some more robust features in the full server product.  These features will enable more seamless integration of a command-line tool with Dataverse.

From your previous post, it sounds like a JDBC interface or Call-Level Interface to TPT may be available. I could research either of those approaches if you were interested in more tightly coupling TPT into Dataverse.

Hope that helps,

Rocco