LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: phourigan

Hi,

I am creating a series of java nodes to perform zip compression/decompression of a collection of files.

I'd like to be able to set the location and filename of the compressed output from a runtime parameter. Is ths possible?


I'm using LAE 4.1.5.


Here's my Java Node:
About half way down I have ToDo comment:
//ToDo: PatrickH, fetch these from Runtime parameters
String compressedFileName = "d:/documents/compressed.zip";


package {{^JavaPackage^}};

import java.lang.*;
import java.io.IOException;
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.util.zip.Adler32;
import java.util.zip.CheckedInputStream;
import java.util.zip.CheckedOutputStream;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import java.util.zip.ZipOutputStream;

import com.lavastorm.brain.io.data.Record;
import com.lavastorm.brain.io.data.RecordInput;
import com.lavastorm.brain.io.data.RecordMetadata;
import com.lavastorm.brain.io.data.RecordOutput;
import com.lavastorm.brain.io.data.SimpleFieldMetadata;
import com.lavastorm.brain.io.data.SimpleRecordMetadata;
import com.lavastorm.brain.io.data.RecordIo.State;
import com.lavastorm.brain.node.AbstractJavaNode;
import com.lavastorm.brain.node.Log.Level;
import com.lavastorm.brain.node.io.NodeOutput;

/**
*
* @author patrick.hourigan
* @version 1.0
*/
public class {{^JavaClass^}} extends AbstractJavaNode
{
private static final int BUFFER = 2048;

public void setup()
{
//check the input and open it
try {
input(0).open();
} catch (IOException e) {
throw new RuntimeException("Error opening input", e);
}

try {
int outputNum = findOutput("CompressedFileName");
if (outputNum < 0) {
throw new RuntimeException("Can't find output CompressedFileName");
}

RecordMetadata metadata = new SimpleRecordMetadata();
metadata.add(new SimpleFieldMetadata("FileLocation", java.lang.String.class));
metadata.lock();

output(outputNum).metadata(metadata);
output(outputNum).open();

} catch (IOException e) {
throw new RuntimeException("Error opening output", e);
}
}

public void processAll()
{
Record record = null;
int outputNum = findOutput("CompressedFileName");
if (outputNum < 0) {
throw new RuntimeException("Can't find output CompressedFileName");
}

NodeOutput output = output(outputNum);
RecordMetadata metadata = output.metadata();
if (null == metadata) {
throw new RuntimeException("Metadata not available for output CompressedFileName");
}

try {
//ToDo: PatrickH, fetch these from Runtime parameters
String compressedFileName = "d:/documents/compressed.zip";

// Create the ZIP stream to the compressed file required.
FileOutputStream dest = new FileOutputStream(compressedFileName);

// Create a checksum based on Alder32 algorithm. Alder32 is lighter
// and more performant than CRC32
CheckedOutputStream checksum = new CheckedOutputStream(dest, new Adler32());

// Create the checksummed stream
ZipOutputStream out = new ZipOutputStream(new BufferedOutputStream(checksum));
byte data[] = new byte[BUFFER];

while((record = input(0).read()) != RecordInput.EOF) {
// Load the file required for each entry in the overall zip
String fileToCompress = (String) record.field(input(0).metadata().find("FileName")) ;
File f = new File(fileToCompress);

FileInputStream fi = new FileInputStream(f);
BufferedInputStream origin = new BufferedInputStream(fi, BUFFER);

ZipEntry entry = new ZipEntry(f.getAbsolutePath());
out.putNextEntry(entry);
int count;
while ((count = origin.read(data, 0, BUFFER)) != -1) {
out.write(data, 0, count);
}
origin.close();
}

Record outputRecord = metadata.newRecord();
outputRecord.field(metadata.find("FileLocation"), compressedFileName);
output.write(outputRecord);

out.close();
} catch (Exception e) {
e.printStackTrace();
}

}

public void cleanup()
{
// Cleanup code here
RuntimeException rte = null;

for (int i=0; i < numInputs(); i++) {
try {
if (input(i)!=null && input(i).state()!=State.CLOSED_STATE)
input(i).close(false);
} catch (IOException e) {
if (rte==null)
rte = new RuntimeException("Error closing input ("+i+").", e);
}
}

for (int i=0; i < numOutputs(); i++) {
try {
if (output(i)!=null && output(i).state()!=State.CLOSED_STATE)
output(i).close(false);
} catch (IOException e) {
if (rte==null)
rte = new RuntimeException("Error closing output ("+i+").", e);
}
}

if (rte!=null) throw rte;

}

public String propertyBase() {
return "ls.brain.node";
}
}

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: stonysmith

In the Python node is a set of methods such as self.getString(name, default) that can be used to access node parameters by name. I don't know if those were implemented in the Java node, but if they are, then you would use that.

On the Parameter Declarations you fill in the "Run Time Property Name" field with a value such as ls.brain.node.Compressor.filename and then use self.getString("ls.brain.node.Compressor.filename" ) to read it.

Semi-Related question: is somone writing API documentation for the Java node?

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: Tim Meagher

First, if you're using a LAE 4.1.5 installation, then the java node isn't supported - although I am aware that we have had some trial usage within MDS/MDA.

It only really got made "experimental" in 4.1.8 (from memory), and will be fully supported once 4.5 comes out.

Stony :
With the 4.5 release, when this becomes fully supported, sample java nodes will be provided, along with a getting started guide to talk you through the basics, and the javadoc API.

The javadoc API released with the java node will essentially be the subset of our API which is required to write java nodes, which mostly deals with all of the classes for using the I/O, logging, properties and such things.

Regarding the original question :
Since this is dealing with unsupported use of the java node in 4.1.5, I can PM you with what you need to get sorted with the runtime properties.
I'm 90% sure that this hasn't/won't change for 4.5, but it's probably better for me to not post this in public forums for unsupported stuff just in case it does.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: phourigan

Hi Tim,

Thanks for the info. I'll try the details provided in the PM for 4.1.5.

In the long run, I'll hold out for v4.5 before making any of this stuff public.

thanks again