LAE

Welcome to the LAE community!  Please feel free to start a discussion in the discussion tab or join in a conversation.

Discussions

Members

Resources

Events

 View Only

  • 1.  Graph node errors at random

    Employee
    Posted 07-13-2017 08:13

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: J4MB0_82

    Hi,

    Background

    One of our clients has been running a Lavastorm graph we have developed for over a year now. They run this graph every month using a new data set every time. This graph has been distributed globally to 20+ locations. As developers of the graph, we have to frequently troubleshoot issues found when the regions run our Lavastorm process. We have so far been able to resolve most of them.

    The Problem

    We have come across an issue which doesn't seem to point to any particular problem i.e. incorrect syntax, wrong file or dodgy data etc, except that some nodes start to run and then all of a sudden one node fails with the following error message:

    Error while starting execution of node
    �Matching.Sorted+Check+with+reversed+strings+inclu ded.StringLevenshtein_1.Split+N+-+N5.Split+N+-+N+++5�
    Error executing node:
    Node :�Matching.Sorted+Check+with+reversed+strings+incl uded.StringLevenshtein_1.Split+N+-+N5.Split+N+-+N+++5� (handle:573c329449fa7d3f) encountered error during initialization

    I check the node and in this case it happens to be a filter node (called Split) within a composite (Called StringLevenshtein) that errors. I examine the filter node and it looks perfectly fine, I then decide to re-run it and then it works. However, when I then proceed to run the remaining nodes in the graph, it errors again on another node with the same type of error. Again, I examine the node and notice it looks fine and then I re-run that node and it works. However, once I try and run all the other nodes it errors again on another node further down for no apparent reason.

    The workaround

    It seems to work if I run each node separately and if this graph only consisted of 10 nodes, we could probably get away with it. This graph however consists of more the 3K nodes and so this workaround is not viable.

    Can someone please help?

    Other notes

    1) I am only seeing this issue with one location so far (let's call this location A), some other locations have been able to successfully run their process without a glitch. However, I have recently received an e-mail from another location showing similar error messages, but cannot confirm its the same issue. It has only been day two, so it early days and I am concerned other locations may face the same problem.

    2) Prior to seeing location A's node error, when starting up their lavastorm, they had an issue where their server farm was not coming up let alone being able to connect to. I killed the lavastorm session via the console, re-started it and added a new server farm (local desktop). The graph seemed to open up and then we started getting hit with the issues described above. Any connection?

    3) Client mentioned that they had upgraded their machine a few weeks ago, still unsure what was upgraded i.e. possibly their OS (Windows). Can find out, but not sure if this would be the root of the cause.

    4) The client's computer crashed multiple times as we ran the Lavastorm process. Memory issue?

    Any assistance on this matter would be much appreciated.

    Thanks

    James


  • 2.  RE: Graph node errors at random

    Employee
    Posted 07-13-2017 08:41

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: stonysmith

    If you at first ran the graph while connected to a server farm, then all the temp files are stored on the server.
    If you then mid-run switch to the local desktop, the temp files are not accessible. That could cause this error.
    In general, if you have a server farm, you should never use "local desktop".

    Outside of that, a couple of quick questions:
    1) What version of LAE?
    2) What are the values of these three settings? (the attached node can be used to find them)
    ls.brain.filecleanup
    ls.brain.controller.brdcleanup
    ls.brain.dircleanup

    LAE_Properties.brn


  • 3.  RE: Graph node errors at random

    Employee
    Posted 07-13-2017 08:54

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: J4MB0_82

    Hi stonysmith

    Thank you for your response.

    I was unable to run the graph as there was no farm (which in itself was odd). After killing the session, I then manually added a new local desktop I was then able to re-run the whole graph again from scratch.

    I shall check on the version when I speak to the client again and find out the values of the three settings.

    What would this information tell me?

    Thank you in advance