LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: rgvenkatesh

Hi,

We are using the HttpNode to issue rest service calls and has set up a 60secs connection timeout on the node.

For example, when the node is processing around 3K requests, sometime the service times out due to server issues (which we are fixing) the entire node halts with a red cross mark and stops the run.

Is it possible to ignore the timeouts and continue with the run and thereby handling the timeouts more gracefully and storing them as an error in the database for the consecutive requests?

The issue now is, when the node times out on the 2999th requests we have to go in manually and do BRD file workaround to pull the processed files and update them in the database which is not a good practice on a soon to be automated job.

Hope I was able to put up this query as clear as possible.

Regards,
Venkatesh

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: gmullin

Is the 60 second timeout coming from the HTTP server and not a value set on the Optional tab of the node? If the timeout comes from the server, you could make the request smaller, like say return 1000 rows at a time. Put the HTTP node in a while loop where you will append each block of 1000 rows returned to a brd file and loop around to the next 1000 rows and eventually will stop when the rows returned eventually is under 1000, meaning you've gotten all data returned to you.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: rgvenkatesh

I am a little confused with the answer, may be I wasn't clear enough with my question.

1. So, the 60 secs connection timeout is set on the Optional tab on the HttpNode after which we get the below error:
Basically our HttpNode client is not getting any response from the Http server in 60 secs (Which the server side is trying to resolve but we want to handle the timeout errors more gracefully instead of stopping the flow)
ERROR: A connection error occurred for PUT request to http://xxxxxxxxxxxxxxxxxxx: Details: java.net.SocketTimeoutException: Read timed out.
Error Code: brain.node.http.transportProtocolError

2. Also, we are already splitting the request records (using hashsplit) upto 20 sets and hitting 20 HttpNodes parallely (we have around 80K requests).

Now, trying once again with the actual question in a different way:
1. What if some of the 20 HttpNodes fail with the Red cross showing error on point 1 which basically halts the control flow. ? Is it possible to make the HttpNode ignore such error and continue with time out error as an response for the remaining requests?

2. When you talk about a BRD file, does it involve manual intervention of pulling BRD file from the STATUS tab on the httpNode and work with it on the BDR file node? Because if it is so, we are not looking for any manual workaround for this as it is going to be fully automated and will be running the BRX graphs from cronTabs.

I am more concerned about an answer to point 1 and to see how best our HttpNode can handle such scenarios automatically, and if it is not possible to handle point 1, would want to see another way out which does not involve manual intervention for the reasons mentioned above.


Thanks,
Venkatesh

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: rgvenkatesh

Any updates on this guys??