LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: GreggB

In nodes that read files, it seems that if you change the input file name, the status of the node does not change. This seems like a bug to me since the output of the node would no longer match the indicated input.

Do you agree?

Thanks,
Gregg.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: mmarinelli

Hello Gregg. This behavior is by design: node status does not change until the node is re-executed, which signifies that the user actually wants to update the graph with the latest modifications. Our mental model for this has been similar to the behavior you see in scripting or programming environments, where the program isn't executed every time a line of code is changed, but rather the user manually triggers program execution when the code is ready.

That said, we could add an option where users could have every individual node edit trigger a status change and attendant refresh of the downstream nodes. Or, we could simply signify on the node icon that a change was made, but that the node itself has not been re-executed. If we added some visual indicator (an exclamation mark instead of a green check, for example) for edited nodes, do you think that avoid the unwanted condition that you described in your post?

Mark

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: stonysmith

I'd vote for the exclamation mark over automatic re-starting of the node.
Sometimes, I have to edit a node multiple times (especially copy-paste operations) before I'm ready for the node to be started. I wouldn't want it to automatically kick off until I was finished editing.

Case in point: copying over the DbUser/DbService fields from another DbQuery node.. that requires two cut/paste operations.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: mmarinelli

Yes, that's why I was thinking that an indicator is better than an automated re-execution: sometimes your logic spans multiple nodes and it can be cumbersome and computationally expensive to have the nodes firing off as each is edited. I'd love to hear some more opinions on this.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: rboccuzzi

I hadn't really thought of that particular behavior before, but I think a "dirty" indicator, that the node has changed since it's last run, would be useful. If it were an option, I would use it.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: Tim Meagher

In general, I like the idea too.
I was actually going to suggest it myself after reading the initial post.
However, after initial thought I realized that this approach has a number of problems that would need to be thought through to prevent confusion and a bad user interface.

Since the BRG stores the graph itself (including these node parameters), and the BRS stores the run state, what happens if you import a BRS?
Or if you open a BRG and the associated BRS is autoloaded?
There isn't any information available to determine whether the execution resulting in the state in the BRS was performed using all of the parameters stored in the BRG.
Should all nodes with status in this case have indicators to say that they may not be up to date - or should it only be if the status was changed after BRE was loaded?

What do you do about nodes downstream from nodes that have been modified - should these all have indicators too, or only nodes that have directly been modified?

The other major complication would be with textual substitution.
Say I do the following:

1. Declare a graph level parameter "Filename"
2. Set this graph level parameter to "C:\tmp\myFile.brd"
3. Create a BRD File node
4. Set the "File" parameter to : {{^Filename^}}
5. Run the node
6. Change the Graph Level parameter Filename to "C:\tmp\myOtherFile.brd"

What should happen in this case?
Technically, the node configuration hasn't changed, since it is still declared the same.
However, what that evaluates to has changed.

A similar situation would exist if you had - for example - parameters using {{^CurrentDate^}}, (or anywhere where you use the rand function).

Doing all of this evaluation to determine if the nodes should have their up-to-date status indicator modified would be quite computationally expensive, and could lead to a much less responsive interface.

Obviously, there are other issues whereby it would be close to impossible for an indicator to be accurate with regard to whether or not the actual status is accurate.
For instance, consider a graph with a whole bunch of nodes, with an acquisition node as the first node.
If you run the graph, then change the file on the operating system where the acquisition node points to, then even though nothing in the graph has changed, the status is no longer technically "current".
If you reran the graph, it would lead to different results.

I think this last case is really less of a concern, and like I said, I like the idea, however the other cases would probably require more thought before such an indicator could be implemented in any sensible manner.


Tim.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: rboccuzzi

Hey Tim, excellent points. I sort of had a bit of a similar thought path myself (but not as thorough). I was thinking of how we have similar issues today; someone runs a set of nodes, and then saves the graph, then clears the status and temp files, and then exits. Next time, they open the graph, the status is no longer accurate; this is the best we can do in that context. I was thinking this would fall into that category, best effort.

Therefore, this "dirty bit" flag would purely represent that the node has been specifically modified after the run status; not vouching for the validity of the status, per se, but just that changes were directly made or not. I think this is still useful. It would be nice if it picked up some of the external changes, either parameters or library base node changes, but I think it has value even without that; and maybe we could make some of those work, but not all....not sure. Thoughts?

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: stonysmith

At this time, if you disconnect an early node, then every downstream node has its pins set to an orange color.

Perhaps if some node ends up having this DirtyBit set, then all of it's dependents could have its pins set to a blue color. This likely could be done within the code that sets the orange color, which seems fast enough.

Set the DirtyBit icon only on the node that was changed, but show the cascade effect thru coloring the descendant pins.
====
Personally.. I would ignore graph level parameter changes. But.. you could set the DirtyBit on any node that USES that changed parameter.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: GreggB

I see that I started quite a discussion (sorry it's taken me a while to get back).

My personal preference would be an option for file acquisition nodes to invalidate the node if the i/p file name was changed. It would be nice if you could actually monitor the file for changes too (the way some text editors do) but that might be asking too much. This could even be a global option rather than at the node level.

I know that the current behaviour is by design, but here's why I think it is wrong: If you have a node with i/p and o/p connectors and you change the i/p connection to a different node, then its connectors turns red and the node is invalidated. To me, changing the file name in an acquisition node is the same, so should produce the same behaviour.

Thanks for considering this,
Gregg.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: aop

Current design where nodes retain green check status even if node has changed is problematic. From time to time it leads to wrong conclusions about the data as it impossbile to keep track on change status of each node in complex graphs in the middle of rapid development. To avoid the problem I frequently re-run the whole graph while working on a project just to make sure that result of each node reflects the latest changes as I cannot leave room for any mistakes. Row counts in my data are often in the millions so re-running the graph all over again is computationally very inefficient but currently the only way for me to make sure I can rely on my analysis. If change indicator or even auto-clear downstream status after change would be available as an option I would definately use them and I'm certain my colleagues share this opinion as well.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: rpigneri

Dear all,

I'm a fan of Stony's suggestion. The balance makes sense to me.

Also, Gregg's suggestion that the input filename(s) for Acquisition nodes behave as "inputs" is also appealing.

I see value in taking both approaches.

Hope that helps,

Rocco