Data360 Analyze

 View Only
  • 1.  Curl command to HTTP node

    Posted 10-09-2024 09:42

    I have this curl command for posting a file to an S3 bucket which works fine, but I would rather use the HTTP node. I couldn't figure out the right syntax as I constantly get bad request back. For example I tried adding to the body something like this:

    key=xyz&bucket=name&policy=xyz

    I also tried with the values in double quotes e.g. key="xyz".

    I've tried pretty much every variation I can think of and can't seem to get it to work. Does anybody have any idea how to put this in the HTTP node and what the Body would look like?

    Curl Command:

    curl -X POST ^
    -F "key=xyz" ^
    -F "bucket=bucketname" ^
    -F "X-Amz-Algorithm=AWS4-HMAC-SHA256" ^
    -F "X-Amz-Date=20241008T215937Z" ^
    -F "X-Amz-Security-Token=xyz" ^
    -F "Policy=xyz" ^
    -F "Content-Type=text/csv" ^
    -F "X-Amz-Credential=xyz" ^
    -F "file=C:/some/path/test-ignore.csv" ^
    "https://s3.us-west-2.amazonaws.com/services.file-validation"



  • 2.  RE: Curl command to HTTP node

    Employee
    Posted 10-17-2024 12:04
    Edited by Adrian Williams 10-17-2024 12:24
      |   view attached

    Hi Jerry,

    Here is a very bare-bones example that sends the multipart data to the httpbin.org/post API endpoint so you can see the data that was sent in the request.

    The main encoding functionality uses some MIT licensed Python code (see attribution in the Transform node script) and a lot of the form-data functionality is hard-coded, so you will need to adapt it to your own needs.

    Please see the attached zipped .lna file (it should work with Anlayze v.3.8.4+).

    Based on #Stonyware



    ------------------------------
    Adrian Williams
    Precisely Software Inc.
    ------------------------------



  • 3.  RE: Curl command to HTTP node

    Posted 11-11-2024 10:16

    Thanks for the data flow Adrian. I did not realize the body had to be created like that. I'm still having problems with getting the upload to S3 to work but I'll spend some more time on how this format is and hopefully get it to work! I appreciate the help!