We currently using the requests package in a Python node rather than the HTTP node to pull back data from an API call that has an unknown number of pages. I would like to move to the HTTP node and it looks like there are Paging options but I can't seem to get it to work.
Here is what the URL looks (I removed the actual URL we use):
https://<URL>?Year=2023&endMonth=12&pageSize=100&pageNumber={pageNumber}
If the response returns entries, we move onto pageNumber=2 and keep doing that until an empty response comes back, at which case I break the loop.
This is what I tried in the Paging group of the HTTP node but it doesn't return anything. Any suggestions?
PagingModel:
Custom
PagingSetupRequest:
pageNumber = 1
pageSize= 100
PagingProcessResponses:
if response.body != None:
pageNumber += 1
Update:
Perhaps this might help somebody else. After having a second look at the documentation I got it to work.
PagingSetupRequest:
pageNumber = 1
request.url.args['pageNumber'] = pageNumber
PagingProcessResponses (Note that count is returned in my call):
count = response.json['count']
if count < 100:
request = None
else:
pageNumber += 1
request.url.args['pageNumber'] = pageNumber