LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: rboccuzzi

The LAL library is going to be updated on the web more frequently than will be officially announced, other than updating that web page. Here will be a thread where you can come look or subscribe to, and we will post a message for each release that is put up on the web, once it is updated. We will give a very brief description of what has been done, as more detail can always be found by looking at the release notes that are posted on the web for the given release.

Cheers
Rich

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 2.0.2 (Compatible with 4.6) has now been released.

Highlights:
- HTTP Node added - Provides an interface for making URL requests to HTTP servers and retrieving response data
- Known bugs fixed

See the release notes for more details. The Release notes and download can be found on our downloads page.

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 2.1.0 (Compatible with 4.6) has now been released.

Highlights:
- Remove Duplicates node added - Removes duplicate records from the input set

See the release notes for more details. The Release notes and download can be found on our downloads page.

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

The R Node 1.1.0 (Compatible with 4.6) has now been released.

Highlights:
- The 1.1 update to R introduces the ability to output images in addition to data frames

See the release notes for more details. The Release notes and download can be found on our downloads page.

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 2.2.0 (Compatible with 4.6) has now been released.
Highlights:
- Output Raw node added - Takes data from a provided field, and writes it to a data file, dealing with internal encoding as appropriate
- Output Static node added - Takes any text that is entered into the text box and writes those contents directly to a file, without interpretation

See the release notes for more details. The Release notes and download can be found on our downloads page.

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 2.3.0 (Compatible with 4.6) has now been released.
Highlights:
- Get MetaData node added � produces an output describing the metadata on the input given, by producing a record for each field in the input, describing the metadata for that field.
- X-Ref MetaData node added - takes 2 inputs and performs a cross-reference against the metadata of the inputs, to find the common fields and types of the two inputs.


See the release notes for more details. The Release notes and download can be found on our downloads page

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 2.4.0 (Compatible with 4.6) has now been released.
Highlights:
- Encrypt node added - Takes an input BRD file and encrypts fields of the user’s choosing using the triple DES algorithm and SHA-1 password hashing technique
- Decrypt node added - Takes an input BRD file with fields encrypted by the Encrypt Fields node

See the release notes for more details. The Release notes and download can be found on our downloads page

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: rboccuzzi

Hi all,

LAL 2.5.0 (Compatible with 4.6) has now been released.

Highlights:

• Quick Stats - The Quick Stats node provides the functionality to run a number of statistical functions over numeric data in the input records. These functions are sum, min, max, average, count, range and null count.
• Convert Tabular XML enhancements - The node now can be configured to support namespaces in the XML, and can handle optional tags in the data.

UPDATE:

LAL 2.5.1 (Compatible with 4.6) has now been released.

Highlights:

• 2 bugs were fixed with the Convert Tabular XML node.

See the release notes for more details. The Release notes and download can be found on our downloads page.

Regards

Rich

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 2.6.0 (Compatible with 4.6) has been released.

Highlights:
� Pivot Table node added - creates a cross tabulation of data, summaries of two criteria, similar to the pivot table functionality in MS Excel.
� Fuzzy Matching nodes enhancements � new �Date� algorithm, fuzzy by days.
� Various bug fixes and help improvements to the Fuzzy Matching, Band by Strata, and URL Query Builder nodes.

See the release notes for more details. The Release notes and download can be found on our download page.

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: rboccuzzi

Hi all,

LAL 2.6.1 (Compatible with 4.6) has now been released.

Highlights:

• 3 bugs were fixed with the Pivot Table node.

See the release notes for more details. The Release notes and download can be found on our download page.

Cheers
Rich

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 2.7.0 (Compatible with 4.6) has been released.

Highlights:
• Change Metadata node added - change a field’s name and data type within the source data; allows the fields to be reordered, and fields to be duplicated as well.
• Bugfix to the Pivot Table node


See the release notes for more details. The Release notes and download can be found on our download page.

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 2.8.0 (Compatible with 4.6) has been released.
Highlights:
� Deep X-Ref node added - extends the X-Ref node to compares the key fields similar to the X-Ref, but for each record from both inputs that match key fields, it does a field by field comparison using all other (non-key) fields. This can be useful when two data sources are expected to be exactly equal or exactly equal in all common (overlapped) fields.

See the release notes for more details. The Release notes and download can be found on our download page..

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 2.9.0 (Compatible with 4.6) has been released.
Highlights:
� Histogram+ node added - The Histogram+ node combines the Histogram and InFlow Bar nodes to create a histogram of the data and a bar chart of it.
� Enhancements to the Convert Tabular XML node.
� Help improvements to the Change Metadata node.

See the release notes for more details. The Release notes and download can be found on our download page.

Attached is a basic demo/example graph showing the new node, please note that you need to install lal1 before the demo graph will run.

Regards

Simon
Attachments:
Histogram+_demo.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 2.10.0 (Compatible with 4.6) is now available for release.
Highlights:
� A new Output Tableau node. The Output Tableau node allows a user to store LAE data in Tableau format to a file. The user can then use Tableau application to view data in the stored file.
� Various issues fixed with HTTP and Convert Tabular XML node


See the release notes for more details. The Release notes and download can be found on our download page.


Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

Highlights:
� A new Output QlikView node. This node takes LAE data and publishes it directly to the QlikView QVX format. This functionality works well in conjunction with our recently added QlikView integration with the LAE as another data viewer (part of 4.6.0), to offer a complete interface, both ad hoc and integrated, with our partner, QlikView.

See the release notes for more details. The Release notes and download can be found on our download page.

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 2.12.0 (Compatible with 4.6) is now available for release.
Highlights:
� A new Input Raw node. Acquires a file and produces an output field containing the raw file contents.
� A new Input Static node. Produces an output field containing raw data entered manually by the user.
� Updates to Output Raw and Output Static, to allow user provided output filenames.

See the release notes for more details. The Release notes and download can be found on our download page.

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 2.12.1 (Compatible with 4.6) has now been released.

Highlights:
4 enhancements were added to the HTTP node.

See the release notes for more details. The Release notes and download can be found on our download page.

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: mmarinelli

Hi all,

LAL 2.13 (compatible with 4.6) has now been released.

Highlights:

• Two new nodes which provide an interface for working with the SAP ERP platform.

See the release notes for more details. The Release notes and download can be found on our download page.

Regards

Mark

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 2.13.1 (Compatible with 4.6) has now been released.

Highlights:
� Issues fix in a previously released LAL node

See the release notes for more details. The Release notes and download can be found on our download page.

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: jed99

FYI - the link to the release notes on the download page is currently broken

Thanks
-Jon

Originally posted by: simon.wilby
					

Hi all,


LAL 2.13.1 (Compatible with 4.6) has now been released.


Highlights:

� Issues fix in a previously released LAL node


See the release notes for more details. The Release notes and download can be found on our download page.


Regards


Simon
			

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Originally posted by: jed99
					

FYI - the link to the release notes on the download page is currently broken


Thanks

-Jon
			

Thanks for the notification Jon - I'll let the website administrator know - in the meantime please find attached the release notes for 2.13.1

Regards

Simon
Attachments:
SAP Nodes Getting Started.pdf
LAL Release Notes.pdf

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 2.14.0 (Compatible with 4.6) is now available for release.

Highlights:
� The FTP Get node Downloads files from a server using the FTP, FTPS, or SFTP protocol. (Compatible with 4.6.1 or newer)
� The FTP Put node Uploads files to a server using the FTP, FTPS, or SFTP protocol. (Compatible with 4.6.1 or newer)
� The ERP nodes now allow date data from an input pin to be correctly transformed to be sent to the SAP system. Issue 5281

See the release notes for more details. The Release notes and download can be found on our download page.

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 2.16.0 (Compatible with 4.6.1) is now available for release.

Highlights:
� The XML Data node - Reads and converts XML data from an input data source into tabular BRD data. (Compatible with 4.6.1 or newer)

See the release notes for more details. The Release notes and download can be found on our download page.

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 2.17.0 (Compatible with 4.6.1) is now available for release.

Highlights:
� The JSON Data node - Reads and converts JSON data from an input data source into tabular BRD data. (Compatible with 4.6.1 or newer)

See the release notes for more details. The Release notes and download can be found on our download page.

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 2.18.0 (compatible with 4.6) is now available for release.

Highlights:
� The Get MongoDB Data node converts Json data output from MongoDB queries to brd tabular format; very similar to the JSON data node. (4.6.1 or newer)
� The Query for MongoDB node performs queries on a MongoDB database and outputs the results as JSON documents.
� The Update for MongoDB performs updates and deletions to documents within a MongoDB database.
� This version of LAL has fixed an issue where the XML Data node would previously error when attempting to read certain XML files. Issue 5417
� The ERP Connector and Extract ERP Metadata nodes now both allow for Logon Load Balancing using a Group/Server connection. Issue 5418
� The ERP Connector node has been enhanced to allow for executing a BAPI function multiple times based on its input data. Issue 5419
� The ERP Connector node now has a new parameter "AdditionalParameterBehavior". This parameter defines the action to take when the node has inputs for scalar, structure or table import parameters which contains fields that do not exist on the corresponding function in the BAPI to execute. Issue 5420
� The ERP nodes have a new parameter called "LargeDecimalAsDouble". This parameter allows the user to specify that even though the data may not fit into a double field, they want it to be output in such a field. Issue 5421

See the release notes for more details. The Release notes and download can be found on our download page.

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 2.19.0 (Compatible with 4.6) is now available for release.

Highlights:
�The Lavastorm Query for Salesforce node queries SALESFORCE.COM objects via SOQL over the SALESFORCE.COM REST API. (Compatible with 4.6.1 or newer)
�The Lavastorm Transact for Salesforce node creates, reads, updates, and deletes records in SALESFORCE.COM objects via the SALESFORCE.COM REST API. (Compatible with 4.6.1 or newer)
�The Net Present Value node calculates the Net Present Value (NPV) of an investment based on the discount rate and a set of future payments and income in the input data set.

See the release notes for more details. the Release notes and download can be found on our download page.

Also please find attached a couple of example graphs - note the Salesforce nodes are working examples, if you configure the run parameters to match your environment then data will be created/updated/deleted. It is your responsibility to remove these records once you have completed the example.

Regards

Simon
Attachments:
query and transact for SALESFORCE examples.brg
Net Present Value wikipedia data example.brg
Lavastorm Analytic Library Release 2 19 Node Overview v1 0.pdf

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

first off - apologies, this update notification passed me by....

The R Node 1.1.1 (Compatible with 4.6) has now been released.

Highlights:
� The 1.1.1 update to R reduces the processing time required when processing large volumes of data. Issue 5394

See the release notes for more details. The Release notes and download can be found on our downloads page.

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 5.0.20.0 (Compatible with 5.0) is now available for release.

This latest release brings all the remaining LAL nodes (up to and including LAL 2.20.1 release) into the LAE 5.0 application.
The ERP Metadata and ERP Connector nodes are now supported on Linux/Solaris installs. User are required to download the applicable binaries and java files from the SAP website and manually install them into the LAE install directories (further details are in the SAP Nodes Getting Started.pdf)


LAL 2.20.1 brought in the SharePoint group of nodes (Note these require the use of SharePoint 2013 api's so in order to use this functionality you must be using SharePoint 2013):
Folder List for SharePoint
Download for SharePoint
Upload for SharePoint


See the release notes for more details. the Release notes and download can be found on our download page.


Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 2.20.2 (Compatible with 4.6) has now been released.

Highlights:
The previous release of lal 2.20.1 included un-versioned LDR jars which can cause conflicts with the LDR nodes shipped with the core LAE product. This release includes versioned LDR jar files. Note that anyone affected will need to replace the previous jar files � this release will not install LDR jar files that have been overwritten when LAL 2.20.1 was installed. Any LDR customers affected by the issue are advised to contact the support team. LAL-694

See the release notes for more details. The Release notes and download can be found on our download page.

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 2.20.3 (Compatible with 4.6) has now been released.

Highlights:
Previous releases of LAL 2.20 have erroneously shipped the laeNodeApi.jar library packaged with JDK 1.7. This has been found to affect the JDBC Query node for customers using JDK 1.6 instances. This release no longer installs the laeNodeApi.jar.

Note that anyone affected will need to install LAL 2.20.3 and then copy their <lae directory>/lib/java/laeNodeApi-1.2.0.jar file to <lae directory>/lib/java/laeNodeApi.jar (overwriting the previous laeNodeApi.jar file). This is required on both desktop and enterprise environments. Any customers unsure on how to do this are advised to contact the Lavastorm support team. LAL-798


See the release notes for more details. The Release notes and download can be found on our download page.

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

We have a couple of prototype nodes available for trial.

Highlights:
The Hive Metadata node queries a Hadoop environment using HiveQL and ODBC to retrieve schema Metadata
The Hive Sample node generates a sample of the data in the specified Hive table (again using an ODBC connection)

For instructions on how to set up the prototype nodes see the relevant Release Notes (one for users on 5.0.x and the other for 4.6.x users)

For customers who do not have any hive ODBC drivers we have some simba drivers available for you to download on our ftp site.

Ftp details:
ftp.lavastorm.com
Username: simba
Password: Simba4LAL

Regards

Simon
Attachments:
hivePrototype46.brg
LAL Hive Prototype 4.6 Release Notes.pdf
hivePrototype50.brg
LAL Hive Prototype 5.0 Release Notes.pdf

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: spalluru

Hi Simon,
It there anyway to un-install the current LAL ( not BRE) and install the new LAL ?
We don't see un-install LAL program after we install LAL on top of BRE
-Sreekanth

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi Sreenkanth,

There is no way to uninstall lal without uninstalling bre, however, each new version of lal will replace the old one - so you can just install over the top of your current lal install.

hope this helps

Simon

Originally posted by: spalluru
					

Hi Simon,

It there anyway to un-install the current LAL ( not BRE) and install the new LAL ?

We don't see un-install LAL program after we install LAL on top of BRE

-Sreekanth
			

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 5.0.21 (Compatible with LAE 5.0.0.1 or newer) and LAL 2.21.0 (Compatible with 4.6) are now available for release.

Highlights:
� The Hive Sample node generates a sample of the data in the specified Hive table.
� The Hive Metadata node queries a Hadoop environment to retrieve schema Metadata.
� The Hive Join node executes a single specified join within the Hive environment

See the release notes for more details. the Release notes and download can be found on our download page.

Also please find attached an example graph as well as some data used for the examples - It is your responsibility to remove these records once you have completed the example.

Customers on 4.6 instances can download the simba hive drivers here

Customers on 5.0.0.1 or newer automatically have the Hive drivers installed and should reference the Lavastorm ODBC Driver for Hive Configuration Guide.pdf to configure their ODBC connection (found in the documentation folder with the lal releases.)

Regards

Simon
Attachments:
Hive Examples.zip

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: John Moore Lavastorm

Hi all,
LAL 5.0.22 (Compatible with LAE 5.0.0.1 or newer) and LAL 2.22.0 (Compatible with 4.6) are now available for release.
Highlights:
• The Output for TIBCO Spotfire node: Outputs LAE data to the TIBCO Spotfire STDF file format, allowing users to work with the output data within the TIBCO Spotfire application.
See the release notes for more details. the Release notes and download can be found on our download page.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 5.0.23 (Compatible with LAE 5.0.0.1 or newer) and LAL 2.23.0 (Compatible with 4.6) are now available for release.

Highlights:

• The Tableau Data Extract API has been updated to version 8.1.2 and as a result Tableau 8.0 or later releases are required to process files produced by this node. The installation now includes the binaries for linux (on 5.0.23.0 only); Tableau does not officially support Red Hat environments at this time so this functionality is released as beta; as a result we cannot guarantee that this will work as designed nor is this supported by any existing maintenance contracts.
• The Send E-mail+ node sends a Unicode-encoded e-mail or group of e-mails with file attachments to a designated set of recipients.

Bugfixes:

• The Http node now includes a new parameter called MaxDataFieldSize which defaults to 50MB. This has been introduced to reduce the occurrence of the �Out of Memory� message that users experience when attempting to process large files. LAL-1230
• The Change Metadata node now allows datetime records to be converted to date records. LAL-1225

See the release notes for more details. the Release notes and download can be found on our download page.

Also please find attached an example graph - You will need to update the run parameters to suit your environment. The AccessibleFileDir is a directory where the graph will save files to and use those files as example attachments.

Regards

Simon
Attachments:
send_e-mail+_example.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 5.0.24 (Compatible with LAE 5.0.0.1) and LAL 2.24.0 (Compatible with 4.6) are now available for release.

Highlights:

• The Folder List for SharePoint 2010 node lists the contents of a SharePoint 2010 folder using the SharePoint 2010 SOAP API.
• The Download for SharePoint 2010 node Downloads file(s) from a specified SharePoint server using the SharePoint 2010 SOAP API.

See the release notes for more details. the Release notes and download can be found on our download page.


Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 5.0.26 (Compatible with LAE 5.0.0.1) is now available.

Highlights:
� Issues fix in a previously released LAL node
� The TERR Connector node
� The Calculate Workdays node

Details of New Functionality:
The TERR Connector node (Powered by TIBCO�) runs an R script on the embedded TIBCO Enterprise Runtime for R engine.
The Calculate Workdays node calculates the number of whole workdays between two dates.

Bugfixes:
The Send Email+ node no longer responds with �Send hello first� when connecting to a SMTP server using TLS Encryption. LAL-1593
The Send Email+ node error received when HtmlMessageBody is set to Field but no field is specified has been improved. The error received now is �Field parameter not defined (HtmlMessageBody)� LAL-1592
The FTP nodes now support the @ character in both the username and password fields. LAL-1326
The HTTP node tooltip for the MaxDataFieldSize parameter has been changed to �Set the maximum size of response content in bytes. This parameter only applies when DataOutputMode is set to "Field". By default the response size is 52428800 bytes (50 Megabytes).� LAL-1800
The MongoDB node has had various enhancements to the help documentation. LAL-1210

See the release notes for more details. the Release notes and download can be found on our download page.

##Edit##
Please find attached example graphs for the new nodes. Note that the TERR Connector node is a premium node and will therefore require a specific node license to execute.

Regards

Simon
Attachments:
TERR Connector_Example Graph.brg
Calculate Workdays Example Graph.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: Wizardous

Hi 5.0.27 link does not work.

http://www.lavastorm.com/resources/s...lytic-library/

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Thanks for letting me know - I'm in the process of applying the installers to the auto update system and once completed will add a notification here.
That should get round the windows installer being unavailable
I've checked the other links for 5.0.27 (linux 32, linux 64 and solaris) which are all valid. I'll pass on that the windows link is invalid to our website administrator.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 5.0.27.0 (Compatible with LAE 5.0.0.1) is now available.

Overview
New in Version 5.0.27.0 of LAL:
� Issues fix in a previously released LAL node
� The Upload for SharePoint 2010 node

Details of New Functionality
The Upload for SharePoint 2010 node Uploads file(s) to a specified SharePoint server using the SharePoint 2010 SOAP API.

Bugfixes
The HTTP node now has �ConnectionTimeout� parameter LAL-710
The Input Static Undefined {{^_Source_Licensed_^}} Parameter message no longer appears when opening the node. LAL-1671/LAL-1404
The Hive Metadata Query node now handles tables with partitioning, which affected tables containing over 200 columns. LAL-1997/LAL-2004
The Hive Join node now process with partitioned tables LAL-2201

See the release notes for more details. the Release notes and download can be found on our download page.
**Please note - the windows link on the main website is invalid - the download however is available when using the lae auto update functionality. We're hoping to have the website link resolved later today.**

Regards

Simon
Attachments:
SharePoint 2010 upload with folder list incl example.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

The R Node 5.0.1.1 (Compatible with 5.0.0.1) has now been released.

this release removes the test nodes from the installation. No other changes have been made.

The Release notes and download can be found on our downloads page.

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 5.0.28 (Compatible with LAE 5.0.0.1) and LAL 5.1.28 (Compatible with LAE 5.1.0) are now available.

Bugfixes:
The TERR connector node has been renamed to the Lavastorm 'Power R' node.

See the release notes for more details. the Release notes and download can be found on our download page.

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 5.0.29 (Compatible with LAE 5.0.0.1 or newer 5.0.x releases) and LAL 5.1.29 (Compatible with LAE 5.1.0) are now available.

Overview
New in Version 5.x.29.0 of LAL:

Details of New Functionality
The Send Email+ node now includes the cc recipients functionality typically found in mail clients.
The HDFS Upload node uploads file(s) to a specified HDFS server using the WebHDFS API.


See the release notes for more details. the Release notes and download can be found on our download page.

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 5.0.31 (Compatible with LAE 5.0.0.1 or newer 5.0.x releases) and LAL 5.1.31 (Compatible with LAE 5.1.0) are now available.
Note � The nodes below and the Power R node are supported on the Windows platform and Linux 64bit platform only, unless otherwise stated; therefore this release is only applicable to those platforms.

Overview
New in Version 5.x.31.0 of LAL, the following nodes within the Statistical and Predictive Analytic Node Pack:

• The Hierarchical Clustering node
• The K-Means Advisor node
• The K-Means Clustering node
• The Linear Regression node
• The Logistic Regression node
• The Predict Linear Regression node
• The Predict Logistic Regression node
• The R Library Package Download node (only supported on Windows platform)

The Statistical and Predictive Analytic Node Pack (Powered by TIBCO�) is Premium licensed.


Details of New Functionality
The Hierarchical Clustering node classifies data into a specified number of clusters. The data are partitioned into a hierarchy of sub-groups. The hierarchy of sub-groups is constructed from the bottom up by clustering the observations such that the distance between observations is minimized at each step.
The K-Means Advisor node recommends the best number of clusters to use when performing a K-Means analysis of the data.
The K-Means Clustering node classifies data into a specified number of clusters. The data are partitioned into k groups such that the 'sum of squares' distance from the points to the assigned cluster centers is minimized. The node uses the Hartigan & Wong algorithm to partition the observations.
The Linear Regression node models data using linear regression allowing identification of data trends.
The Logistic Regression node models data using logistic regression allowing identification of data trends.
The Predict Linear Regression node uses a linear regression model to predict the value of a dependent variable based on the specified values of the independent variables.
The Predict Logistic Regression node uses a logistic regression model to predict the probability of a successful outcome of a dependent variable based on the specified values of the independent variables.
The R Library Package Download node downloads the primary 'Comprehensive R Archive Network' (CRAN) R Library packages that are prerequisites for the correct operation of the nodes within the Statistical and Predictive Analytics Node Pack.

There are also example graphs attached, you will need as a minimum a trial Statistical and Predictive Analytic Node Pack license to execute these examples.

See the release notes for more details. the Release notes and download can be found on our download page.

Regards

Simon
Attachments:
Hierarchical_Clustering_Example_v0.1.brg
KMeansAdvisor_example_0.1.brg
KMeansClustering_example_0.1.brg
Linear Regression example graph 0.1.brg
Logistic regression example graph 0.4.brg
Predict Linear Regression Example v0.1.brg
Predict Logistic Regression example v0.1.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

LAL 5.0.32 (Compatible with LAE 5.0.0.1 or newer 5.0.x releases) and LAL 5.1.32 (Compatible with LAE 5.1.0) are now available.

Overview:

• The HDFS Download node
  
• The FTP nodes (Get and Put) enhancement

Prerequisites
To utilize the HDFS nodes you must have a working instance of a Hadoop Hive Cluster.

Details of New Functionality
The HDFS Download node Downloads file(s) from a specified HDFS server using the WebHDFS API.
The FTP Put and FTP Get nodes now include the following two new parameters (LAL-3556):

• The �PassiveMode� parameter to support Active and Passive FTP modes.
• The �ConnectionTimeout� parameter to deal with no response scenarios.

An example graph for the HDFS Upload and Download nodes is attached, in order for the nodes to work the user will need to populate the relevant run parameters.

See the release notes for more details. the Release notes and download can be found on our download page.

Regards

Simon
Attachments:
HDFS_Upload_Download_Example.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: StuartOboler

Hi,
I've just set up a new user with lavstorm.
Is LAL 5.1.33 compatible with BRE 6 ?

I don't want to set it up if it's going to cause problems and result in a negative experience.

Cheers,
Stuart

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi,

LAE 6.0 comes shipped with lal 6.0.33.
LAL 6.0.33 brings all the functionality of lal 5.1.33 into LAE 6.0

but to clarify the compatibility question:

• lal 5.1.x is compatible only with LAE 5.1 installations
• lal 6.0.x is compatible only with LAE 6.0 installations

If you install a non compatible version (ie lal 5.1 on lae 6.0) you will break functionality.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

Note � The nodes below and the Power R node are supported on the Windows platform and Linux 64bit platform only, unless otherwise stated; therefore this release is only applicable to those platforms.

The following nodes are within the Statistical and Predictive Analytic Node Pack (Powered by TIBCO�) which is Premium licensed.

New in LAL 5.1.33 / LAL 6.0.33
Overview:

• The Market Basket Analysis node
• The Market Basket Miner node

Details of New Functionality:
The Market Basket Analysis node generates Frequent Itemsets and Association Rules from transactional data.
The Market Basket Miner node Extracts association rules from an association rule model.


New in LAL 5.1.34 / LAL 6.0.34
Overview:

• The Time Series Forecast node
• The Time Series Diagnostics node

Details of New Functionality:
The Time Series Forecast node uses the Holt-Winters method to forecast a time series that can optionally contain a trend and seasonal variations.
The Time Series Diagnostics node generates plots of univariate time series data.


New in LAL 5.1.35 / LAL 6.0.35
Overview:

• The Decision Forest node
• The Predict Decision Forest node

Details of New Functionality
The Decision Forest node models data using the Random Forest model allowing identification of data trends using an �ensemble� of decision tree models.
The Predict Decision Forest node predicts the value or classification for a dependent variable in a Random Forest model based on the value of the independent variables.


Example graphs are attached.

See the release notes for more details. the Release notes and download can be found on our download page.

Regards

Simon
Attachments:
Time Series Diagnostic Example v0.1.brg
time series forecast example v0.2.brg
Market_Basket_Examples_Groceries_v0.1.brg
Predict Decision Forest Node example graph v0.1.brg
Decision Forest Node example graph_v0.1.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

Note � The nodes below and the Power R node are supported on the Windows platform and Linux 64bit platform only, unless otherwise stated; therefore the new nodes are only applicable to those platforms. The releases do include bugfixes for other nodes so it is still worth installing this latest version.

New in LAL 5.1.36 / LAL 6.0.36
Overview:
The following nodes are within the Statistical and Predictive Analytic Node Pack (Powered by TIBCO�) which is Premium licensed.

• The Linear Regression Diagnostics node
• The Logistic Regression Diagnostics node

Details of New Functionality
The Linear Regression Diagnostics node generates diagnostic plots from a linear regression model.
The Logistic Regression Diagnostics node generates diagnostic plots from a logistic regression model.

Bugfixes:
The Output Excel and Append Excel nodes FormatOutput parameter has been updated to include a �From Template� option; this allows a template file to specify the cell formatting.
Previously the FormatOutput parameter only provided a True/False option.
In addition, two further options have been included; �From BRD� (which behaves in the same way as the previous �True� Boolean option) and �No Formatting� (which behaves in the same way as the previous �False� Boolean option).
The True/False options remain so that users with existing graphs can continue to run their graphs without any impact. LAL-2078.

LAL 5.1.36.1 brings in the Output Tableau node fix to work in the same way as the Tableau data viewer within LAE; this change introduces a performance improvement for the node. LAL-1817

New in LAL 6.0.37

Overview:

• The HDFS Upload node enhancement

Details of New Functionality
The HDFS Upload node previously had a file size limit of 256MB, the node now supports the uploading of large files (in excess of 1GB).

Bugfixes:
The �SortInput� parameter for the Agg Ex node has been moved to the General tab in line with other nodes LAL-5029
The Output Tableau node has been improved to work in the same way as the Tableau data viewer within LAE; this change introduces a performance improvement for the node. LAL-1817

Example graphs are attached.

See the release notes for more details. the Release notes and download can be found on our download page.

Regards

Simon
Attachments:
Logistic Regression Diagnostics Example v0.2.brg
Linear Regression Diagnostics Example v0.1.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

New in LAL 6.1.38 / LAL 6.0.38

• The Lavastorm Get for Salesforce node
• The HTTP node enhancement
• The K-means Advisor node enhancement
• The K-means Clustering node enhancement
• The R Library Package Download node enhancement

Details of New Functionality

The Lavastorm Get for Salesforce node queries SALESFORCE.COM objects via SOQL over the SALESFORCE.COM REST API. We recommend all new graphs use the �Lavastorm Get for Salesforce� node instead of the existing �Lavastorm Query for Salesforce� node which has been deprecated. The �Lavastorm Query for Salesforce� node will continue to operate for existing graphs.

The HTTP node has been enhanced to now include two �Truststore� parameters. These parameters enable the node to utilize a user specified Java Runtime Environment (JRE) trust store. LAL-4928

The K-Means Advisor node has been enhanced with a new �TransformData� parameter which provides the option to apply z-score standardization to each field in the input data. LAL-2306

The K-Means Clustering node has been enhanced with a new �TransformData� parameter which provides the option to apply z-score standardization to each field in the input data. LAL-2217

The R Library Package Download node has been amended to use our own repository https://lavastorm.exavault.com/p/LavastormStatRepo. This is due to compatibility issues with packages distributed on the official CRAN sites. Users of the R Library Package Download node do not need to do anything different. Users of the Power R node, who do not have the R Library Package node, are expected to use the above location in line with the install.packages command and save the libraries to <LAE temp directory>/TERR/libs/. The Lavastorm repository does not currently support linux distributions. The source files can be viewed here LAL-5379

Experimental Nodes
WARNING: A range of experimental nodes have been included with the LAE product that streamline a range of common data acquisition, preparation and analysis tasks, particularly for business users. When using nodes marked as experimental some features may not work. The name of the node or parameters may change, causing any graphs that use them to require updating. The documentation may be incomplete. Use of these nodes implies a willingness to accept these limitations.

• Change Fields - Changes the name and/or data type of selected fields in the input data. Selected fields are reordered
• Cleanse String Fields - Removes or replaces non-alphanumeric characters and reduces consecutive whitespace to a single space
• Compare Data Sets - Compares the values for a specified field in each of the configured (up to 3) data sources
• Compare Multiple Data Sets - Compares the values for a specified field in each of the configured data sources
• Complex Lookup - Provides complex lookups using logical operator-based rules
• CSV Data - Imports data from a CSV file or comma-separated data from an input field
• Date Range Filter - Filters data using a date range
• Date Range Lookup - Provides date-related lookups using logical operator-based rules
• Delimited Data - Parses delimited data from a specified file, a set of input files or from data fields in the input
• Dynamic Split String - Splits a string type field using a specified delimiter character
• Excel Options - Reads and converts a single-sheet Microsoft Excel (.xls, .xlsx) file for input
• Filter By Date Range - Filters data by data range
• Filter By Dates Within Time Period - Filters data to identify records that are within a specified time period of a reference date (or today)
• LAE Properties - Retrieves LAE properties information
• Load CSV File with 1 Additional Column - Reads and converts a CSV file that has an additional column
• Make R-Safe Fieldnames - Changes field names to valid R variable names
• More/Less/Equal Date Filter - Filters data by comparing the values in a date field agaist a specified date
• Multi-Format Date Conversion (with 2D checks) - Analyzes and normalizes dates in a field to have a consistent date format
• Multi-Format Date Conversion - Analyzes and normalizes dates in a field to have a consistent date format
• Multiple Way Data Profiling - Analyses the contents of up to four input data sets (e.g. database tables) to identify fields that may possibly contain primary keys and fields that contain foreign keys
• N Period Date Filter - Filters the data by checking the value in a date field to determine if it is within a specified 'window' time period of a specified date or today's date
• Oracle tnsnames.ora Analyzer - Extracts information from an Oracle tnsnames.ora file
• Power R Read Delimited File - Imports data from a delimited file
• Power R Read SAS Data File - Reads and converts a SAS Database (.sas7bdat) file for input
• Power R Stats Over Calendar Period - Calculates statistics for a time series over calendar periods
• Quick Acquisition - Imports data from a CSV or Excel file
• Rolling Statistics - Analyzes data to generate statistics for a rolling 'window' of records
• Select Fields - Filters the fields (columns) in the input data
• Split Field By Delimiters - Splits string data by delimiter into multiple columns
• Split String - Splits the contents a specified field using a given delimiter string
• String Based Lookup - Provides lookups on string variables using logical operator-based rules.
• Three Way Data Profiling - Analyses the contents of three input data sets (e.g. database tables) to identify fields that may possibly contain primary keys and fields that contain foreign keys
• Top/Bottom N Records - Extracts the top 'N' or bottom 'N' records from the data using the value of a specified field as the selection criterion
• Two Way Data Profiling - Analyses the contents of two input data sets (e.g. database tables) to identify fields that may possibly contain primary keys and fields that contain foreign keys

See the release notes for more details. the Release notes and download can be found on our download page.

Regards

Simon

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

New in LAL 6.1.40 / LAL 6.0.40

Overview
New in Version 6.x.40 of LAL:

• The Predict Quantile Regression node (within the Statistical and Predictive Analytic Node Pack)
• The Lavastorm Get for Salesforce node enhancement
• The Lavastorm Query for Salesforce node (un-deprecated)

The Statistical and Predictive Analytic Node Pack (Powered by TIBCO�) is Premium licensed.
Note � The Predict Quantile Regression node is supported on the Windows platform, RedHat and SUSE Linux 64bit platform only.

Details of New Functionality

The Predict Quantile Regression node predicts the value of a dependent variable in a Quantile Regression model based on the value of the independent variables.
The Lavastorm Get for Salesforce node has been enhanced to return column names when the query returns no records. LAL-4726
The Lavastorm Query for Salesforce node has been un-deprecated; the node will be deprecated in a future release when all additional features are delivered in the Lavastorm Get for Salesforce node.

Bugfixes
The Send Email+ node now supports mixing the ToAddress and MessageBody parameters with literal or field options; previously the node would overwrite the MessageBody �Field� setting when the ToAddress was set to �literal�. LAL-4382

known issues/considerations
The Lavastorm Update for Salesforce node does not currently handle null entries for fields to be updated. LAL-5538

New in LAL 6.1.39 / LAL 6.0.39

Overview
New in Version 6.X.39 of LAL:

• The Lavastorm Metadata for Salesforce node
• The Lavastorm Update for Salesforce node
• The Quantile Regression node (within the Statistical and Predictive Analytic Node Pack)

The Statistical and Predictive Analytic Node Pack (Powered by TIBCO�) is Premium licensed.
Note � The Quantile Regression node is supported on the Windows platform, RedHat and SUSE Linux 64bit platform only.


Details of New Functionality

The Lavastorm Metadata for Salesforce node retrieves metadata in SALESFORCE.COM objects via the SALESFORCE.COM REST API.
The Lavastorm Update for Salesforce node inserts, updates, upserts or deletes records in SALESFORCE.COM objects via the SALESFORCE.COM REST API.
The Quantile Regression node models data using quantile regression allowing identification of data trends for conditional quantiles of a response variable distribution.

Bugfixes
The Excel File node default for the NoOutputPinErrorBehaviour parameter has been amended to log instead of Error LAL-5238 (LAL 6.1.39 only)
The Lavastorm Get for Salesforce node now supports queries for parent-to-child and child-to-parent relationships. LAL-5282
The Lavastorm Transact for Salesforce node requested a new session on each iteration, this resulted in BRE hanging if more than 10 instances were running. The new suite of Lavastorm for Salesforce nodes (Get, Metadata and Update) all output the session details so the same session can be used repeatedly. LAL-5077
The HTTP node now masks the TrustStoreFilePassword parameter LAL-5278


See the release notes for more details. the Release notes and download can be found on our download page.

Example graphs attached.

Regards

Simon
Attachments:
SALESFORCE_get_update_metadata_example_v0.1.brg
Predict Quantile Regression Example v0.2.brg
Quantile Regression Example v0.2.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

New in LAL 6.1.41 / LAL 6.0.41

Overview

• The Cleanse String node enhancement
• The Data Converter node (Experimental)
• The Lavastorm Get / Metadata / Update for Salesforce node enhancement
• The Select Fields node enhancement
• The Spark Download node (Experimental)
• The Spark End Session node (Experimental)
• The Spark Java Submit node (Experimental)
• The Spark Sort node (Experimental)
• The Spark Start Session node (Experimental)
• The Spark Upload node (Experimental)

Details of New Functionality

The Cleanse String node has an improved performance. LAL-5643
The Data Converter node converts the data type of each field based on the discovered data type.
The Lavastorm Get / Metadata / Update for Salesforce nodes now support the use of a Proxy server. LAL-5556 / LAL-5557 / LAL-5558.
The Select Fields node has an improved performance. LAL-5644
The Spark Download node downloads data in the Avro file format from a Spark cluster.
The Spark End Session node ends a Spark session.
The Spark Java Submit node provides an interface with a Spark cluster to execute a Spark application.
The Spark Sort node sorts data in the Avro file format using a Spark cluster.
The Spark Start Session node starts a Spark session.
The Spark Upload node uploads data to a Spark cluster in the Avro file format.

Bugfixes
The Output Excel and Append Excel nodes now cater for dates greater than the year 2098. LAL-5563
The Change Metadata node now caters for dates greater than the year 2098. LAL-5507

Third party known issues/considerations

• The Avro 1.7.7 specification calls out some constraints on the metadata that it can support. Specifically it places restrictions on the names of fields:
  The Field names must:
  start with [A-Za-z_]
  subsequently contain only [A-Za-z0-9_]
• Avro 1.7.7 does not support date, time and datetime data types. As a result if you wish to upload data and use the Lavastorm nodes then these fields will need to be converted to string.

See the release notes for more details. the Release notes and download can be found on our download page.

Example graphs attached for the Spark based nodes. Please read the Spark Configuration v0.1.pdf before attempting to run the nodes.

Regards

Simon
Attachments:
Spark Configuration v0.1.pdf
spark_java_submit_node_example_v0.1.brg
Spark_Sort_Node_Example_v0.1.brg
SparkSortExample-1.0.0.zip

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

New in LAL 6.1.42 / LAL 6.0.42

Overview

• Tibco Enterprise Runtime for R (TERR) embedded R engine has been updated (6.1 only)
• The Spark Filter node (Experimental)
• The Spark Java Submit node enhancement
• The Spark Sample node (Experimental)

Details of New Functionality
The Tibco Enterprise Runtime for R (TERR) embedded R engine has been updated from 3.2 to 4.0 on both the Windows 64bit and Linux 64bit platforms. LAL-5550 (6.1 only)
The Spark Filter node filters data using a Spark cluster.
The Spark Java Submit node has been enhanced to allow customers to specify configuration properties for the Java application execution. LAL-5791
The Spark Sample node samples data using a Spark cluster.

Bugfixes
The Lavastorm Update for Salesforce node now advises which record has failed to update and stops executing at that record. LAL-5537
The Lavastorm Update for Salesforce node now completes updates where values are null. LAL-5538

See the release notes for more details. the Release notes and download can be found on our download page.

Example graphs attached for the Spark based nodes. Please read the Spark Configuration v0.1.pdf before attempting to run the nodes.


Regards

Simon
Attachments:
Spark_Sample_Node_Example_v0.1.brg
Spark_Filter_Node_Example_v0.1.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

New in LAL 6.1.43 / LAL 6.0.43

Overview

• The Lavastorm Get for Salesforce node enhancement
• The Spark Agg node (Experimental)
• The Spark Data Viewer node (Experimental)
• The Spark Join node (Experimental)
• Spark nodes enhancement

Details of New Functionality
The Lavastorm Get for Salesforce node now supports the SOQL convertCurrency and date functions. LAL-5871
The Spark Agg node aggregates data using a Spark cluster.
The Spark Data Viewer node previews data from Spark.
The Spark Join node joins data using a Spark cluster.
A number of Spark based nodes (i.e. those that include an InputSource parameter) have been enhanced to provide a default value for the InputSource parameter; this is so that users only need to connect the second output pin from a previous Spark node into the second input pin. LAL-5843

Bugfixes
The TIBCO Enterprise Runtime for R (TERR) based nodes (the Statistical and Predictive Analytic Node Pack) were failing to execute on the Windows 32bit operating system. The TERR remoteJvmArch property has been updated to specify the operating system bitness instead of the default 64bit option. LAL-5950 (LAL 6.1.43 only)

See the release notes for more details. the Release notes and download can be found on our download page.

Example graphs attached for the Spark based nodes. Please read the Spark Configuration v0.1.pdf before attempting to run the nodes.


Regards

Simon
Attachments:
Spark_Agg_and_Data_Viewer_Example_v0.1.brg
Spark_Join_Example_v0.1.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: simon.wilby

Hi all,

New in LAL 6.1.45 / LAL 6.0.45

Overview
Please Note: All of the Spark nodes now require Spark 1.5.0 as a minimum in the <LAE Install>/lib/java/lal/spark directory. In addition the Spark SQL Query node requires the Hive Client Configuration to be located on the laeServer machine. (See the Spark Configuration Guide for Lavastorm.pdf included in your installed documentation).

• The Archive Compress node
• The QVD File node
• The Spark SQL Query node (Experimental)
• The Spark Agg node enhancement
• The Spark Join node enhancement
• The Excel File node enhancements

Details of New Functionality
The Archive Compress node creates a compressed archive file containing one or more files. LAL-4961
The QVD File node imports data from a QVD file. LAL-3102. Additional software libraries are required in order for this node to work (commons-io-2.4.jar, irbi-fileutils-0.1.0.jar, jdbm-3.0-SNAPSHOT.jar, qvd-converter-core-1.5.0.jar and qvd-reader-1.5.0.jar).
Please contact Ralf Becher using the contact details below in order to purchase these libraries. Once you have those libraries they need to be saved to <LAE install Directory>/lib/java/lal.
IT Beratung Ralf Becher
Leipzig, Germany
ralf.becher@web.de

The Spark SQL Query node provides an interface with a Spark cluster to execute a Spark SQL query. LAL-6194
The Spark Agg node has new parameters included to allow customers to include or exclude fields from the aggregate function. LAL-5817
The Spark Join node has been enhanced to include new parameters so that users are able to filter on one or both datasets. LAL-6001
The Excel File node has a new parameter called IgnoreCellFormatting which allows the raw data underneath cell formatting to be imported to LAE. The default behavior is set to �Ignore� to allow for backwards compatibility. LAL-1554/gnats 5119

Bugfixes
The Trim Fields node now fully caters for fields with or without single quotes. LAL-5004
The Trim Fields node set to NULL if empty option has been improved to work on all fields. Also if the field names do not exist for the excluded/included option an exception is now returned. LAL-4049

Third party known issues/considerations
The Spark SQL Query node has highlighted some Apache issues with the following scenarios:

• Selecting a field with binary type fails with the exception �UnresolvedUnionException: Not in union ["bytes","null"]�. This is already raised on Apache JIRA: https://issues.apache.org/jira/browse/AVRO-1401
• Describe operation returns 3 fields (col_name, col_type, comment) however the comment field is handled as not �nullable� but returns a NULL value.

The Spark SQL Query node processes against Hive tables. When Hive tables are processed by the cluster, the minimum memory requirement is higher compared with the memory required to run other Spark nodes that do not access Hive. The DriverMemory and ExecutorMemory both have a minimum 5G threshold. Users are advised to increase this for larger environments.


New in LAL 6.1.44 / LAL 6.0.44

Overview
Please Note: All of the Spark nodes now require Spark 1.5.0 as a minimum in the <LAE Install>/lib/java/lal/spark directory (See the Spark Configuration Guide included in your installed documentation).

• The SharePoint 2010 suite of nodes enhancements
• The Spark Agg node enhancements
• The Spark Filter node enhancements
• The Spark Java Submit node enhancement
• The Spark Join node enhancement
• New Guides included

Details of New Functionality
The SharePoint 2010 suite of nodes have been enhanced to disable Forms Based Authentication. LAL-5739
The SharePoint 2010 suite of nodes have been enhanced to support situations where there is a URL-Folder Name mismatch. Please note: If there is a mismatch then additional processing is required to match up display names with underlying names used by the API. LAL-5623
The Spark Agg node has been enhanced to include the function name in the output columns. LAL-6112
The Spark Agg node now includes grouping functionality. Users are able to group aggregated data by one or more fields so that any records with identical values in the specified group fields are combined into a single record. LAL-5818
The Spark Filter node now allows a user to specify which fields are to be included or excluded on the output file(s). LAL-5814
The Spark Filter now includes the ability to filter on an additional two sets of filter criteria. LAL-5813
The Spark Java Submit node has a new parameter called SparkAssemblyJar. Users are now able to specify the JAR they wish the Spark application to execute with. Users are advised to review the �Spark Configuration Guide for Lavastorm.pdf� located in their LAE installation. LAL-6075
The Spark Join node now allows the ability to select from an additional four join types (default join type �Inner�): Left Outer, Right Outer, Full Outer, Left Semi. LAL-5998
Three new documents have been added to the install of LAL. The documents included are: �CRAN Package Install Guide for Lavastorm.pdf�, �Spark Configuration Guide for Lavastorm.pdf� and �SSL Connections Guide for Lavastorm.pdf�. LAL-6111
LAL V6.1.45 Release Notes � Lavastorm Analytics 2016 All Rights reserved Page 17

Known issues/considerations

• The Spark Sample node has been updated to return �N Records� instead of �First N Records�. When using the 'N records' option, the order of the generated sample set is not guaranteed to reflect the order of the data in the input file(s).

Attached are example graphs for the Archive Compress and Spark SQL nodes

Regards

Simon
Attachments:
LAL_Archive_Compress_Node_Example_v0.1 (1).brg
Spark SQL Query Example v1.0.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: awilliams1024

Hi All,

LAL 6.1.48 (Compatible with LAE 6.1.x) is now available.

New in LAL 6.1.48

Overview

• Data Converter node GA
• Spark Agg node enhancement
• Spark nodes Kerberos keytab authentication enhancement
• Spark Download and Spark Data Viewer string data type enhancement
• Spark nodes delimited file support enhancements
• Spark nodes GA

Details of New Functionality
The status of the (previously Experimental) Data Converter node has now been changed from experimental to fully supported.

The Spark Agg node now outputs a GroupBy field when the field is also included in an aggregation function. LAL-6480

The Spark nodes have been enhanced to offer an additional Kerberos authentication method which uses a secure keytab file.

The Spark Download node and Spark Data Viewer node have been enhanced to enable the customer to specify whether string fields should be output as unicode field type (default) or as string field type. LAL-6870
A number of Spark nodes have been enhanced to allow customers to specify a delimited file as the node’s input data source instead of the default Avro file format. Configuration options enable the customer to specify: the delimiter character, whether the delimited file has a header record, the quote character, the escape character and whether empty strings are to be considered NULL values. By default, the nodes attempt to automatically identify the schema. The schema can also be explicitly defined. LAL-6805, LAL-6697

The status of the (previously Experimental) Spark nodes has now been changed from experimental to fully supported. This relates to the following nodes: Spark Start Session, Spark End Session, Spark Upload, Spark Download, Spark Data Viewer, Spark Agg, Spark Filter, Spark Join, Spark Sample, Spark Sort, Spark SQL Query.

Bugfixes
Some extraneous parameters are no longer visible on the JSON Data node and the Logistic Regression Diagnostics node. LAL-6523, LAL-6840
Duplicate help text was removed from the Spark SQL Query node’s help information. LAL-6797

Removed Functionality
The following experimental nodes have been removed from the installation:

• Complex Lookup
• Date Range Lookup
• String Based Lookup

Third party known issues/considerations
In the situation where a Spark node is configured to import data from a delimited file and the schema has been defined by the user, the schema must be valid. When the schema is invalid the node execution may complete successfully but no records will be output.

The Release Notes and installers can be found on our download page.

Regards,
Adrian

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: sobhsepid

X-Ref MetaData node added - takes 2 inputs and performs a cross-reference against the metadata of the inputs, to find the common fields and types of the two inputs.