Data360 Analyze

Using Data360 - 3.6.8. Is there a way to configure the HTTP node to ignore errors and process al the incoming records? I am trying to execute an address validation process and send the URL data to the vendor. The process error but I was looking to have the process complete all 127K records and indicate which where successful and which ones errored.

The error I am getting which stops the process is 

The URL provided cannot be converted to a URI. Cause: Illegal character in query at index 177

Some retry capabilities were recently added to the HTTP node but these were intended to overcome intermittent network issues rather than hard faults with the incoming data.

You could validate the contents of a query string in the URL and filter out URLs with invalid characters prior to passing the remaining URLs to the HTTP node (see attached example data flow).

Alternatively, you may be able to put the HTTP node into a child data flow and then use the Execute Data Flow node to run the child data flow, passing in the URL to the child data flow as a run property. The 'FailedDataFlowBehavior' property would be set to log or ignore failed runs.

Attached files

Validate_URL_Query_String - 15 Sept 2021.lna

Note, the Retry and Exception Behavior property options that were configured on the HTTP node in the example data flow have no effect on whether the node would error if there was an invalid character in the input query string, they are just a hang-over from some of the testing.