LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: dkhuon@gogoair.com

Has anyone done an FTP synch mode where the transfer will not take place unless the source file either does not exist or its creation/modified date is newer than an existing file in the target folder.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: mlowe

There is an FTP node (perhaps in prototype - let me look this up) that allows you to get a list of files and their sizes, you could compare an old list you retrieved against the latest list and then only get the files that have changed. Let me know if that will work, I know I have that node floating around somewhere?

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: dkhuon@gogoair.com

Originally posted by: mlowe
					

There is an FTP node (perhaps in prototype - let me look this up) that allows you to get a list of files and their sizes, you could compare an old list you retrieved against the latest list and then only get the files that have changed. Let me know if that will work, I know I have that node floating around somewhere?
			

It would come in handy. Please let us know where to get it, once you find it. Thanks for your help.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: mmarinelli

You'll definitely want to check out the FTP Put node available in the Lavastorm Analytics Library add-on available for download at this link. It includes a FileExistsBehavior parameter which governs the behavior of writes/overwrites based upon file existence/timestamp. I think it should do what you're looking for.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: dkhuon@gogoair.com

My problem is that my job runs on the server receiving data from outside. I would not be able to trigger the transfer using Ftp Put. I am waiting for mlowe's solution (see thread above.) Thanks anyway for your suggestion. I am sure I will be using this feature in another situation, in the future.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: mlowe

Hi, I've attached a simple ftp list node that allows you to select between LIST, NLST and MLSD (ListMethod parameter) where LIST is not always the same across implementations, but should be ok if you are using it on a single server with limited chance of ftp server upgrade. The MLSD option allows for a machine readable output to be returned and I've put an example that pulls the filename, size and modified date/time from this list for reference (although the "field" should be "FileName" not "List" as it is in the attached graph).

This should allow you to generate a list and/or store it to compare against a new list at some stage in the future. Then you would PUT/GET only those files that changed.

Let me know if you have any questions/issues? (was tested on 4.6.1.0)
Attachments:
ftp-list.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: dkhuon@gogoair.com

Originally posted by: mlowe
					

Hi, I've attached a simple ftp list node that allows you to select between LIST, NLST and MLSD (ListMethod parameter) where LIST is not always the same across implementations, but should be ok if you are using it on a single server with limited chance of ftp server upgrade. The MLSD option allows for a machine readable output to be returned and I've put an example that pulls the filename, size and modified date/time from this list for reference.


This should allow you to generate a list and/or store it to compare against a new list at some stage in the future. Then you would PUT/GET only those files that changed.


Let me know if you have any questions/issues?
			

Hi,

two quick questions:
1. what version of Lavastorm will this graph runs: we have 4.6.0.
2. how do I extract/download this attachment from the Forum (I am a newby here). I tried to do "Save Target as.." on the link, and got no effect. I can wait until I am back at the office tomorrow, though.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: ejones

For the second of these two questions, I ran into a trouble downloading files as soon as I upgraded to the latest IE. The work around turned out to be to rename the file that was downloaded to the correct name with the .brg extention. The other work around is to use Google Chrome or FireFox instead of IE, or, with great difficulty, you can downgrade to an earlier version of IE like I eventually ended up doing.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: dkhuon@gogoair.com

It's my fault: it was my insanity in keeping on clicking the attachment icon... and expect a different outcome :-(

Instead of clicking the real ftp_list.brg link. It now works fine, either with chrome browser, or my current IE 10.

Thanks so much for your help.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: dkhuon@gogoair.com

Originally posted by: dkhuon@gogoair.com
					

It's my fault: it was my insanity in keeping on clicking the attachment icon... and expect a different outcome  :-(


Instead of clicking the real ftp_list.brg link.  It now works fine, either with chrome browser, or my current IE 10.


Thanks so much for your help.
			

I have received several replies, suggestions and source codes for my problem. I ended up using all these sources (Ftp LIST and MDTM), mixed them together a bit, and added a few extra try / catch code blocks in Python to control the aborts. Everything works well for me.

Again, thanks so much for your helping hands. Here's this little emo to show my appreciation to you all:

Note: my feeling (or hunger) for) an ability to control node abort (see my other post) from outside the node still remain. Hope Lavastorm will grant us this wish some day in a near future.