HTTP PUT to AWS S3 - Payload Issue

wshirley · Sat Sep 14, 2019 11:38 am

I'm trying to upload a file to an AWS S3 bucket. I've come to a dead end for the past several days and haven't been able to find any online resources to address my current challenge.

I've successfully written the code to calculate the signature and Authorization Header. I'm using the following headers and ifURLTransfer commands:

    m.SearchUrl.SetRequest("PUT")
    PutStr = "/Myfile.txt HTTP/1.1"
    m.SearchUrl.AddHeader("PUT", PutStr)
    m.SearchUrl.AddHeader("Host",Host)  'Host format = bucket-name.s3.amazonaws.com
    m.SearchUrl.AddHeader("Date", DateString)  'Date format = Sat, Sep 14, 2019 18:14:25 GMT
    m.SearchUrl.AddHeader("Authorization", AuthorizationHeaderString) 'for example: AWS4-HMAC-SHA256 Credential=AKIAxxxxxxxxxxxxxxxx/20190914/us-east-1/s3/aws4_request,SignedHeaders=date;content-length;content-type;host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-storage-class,Signature=3af54f13f05222daff50047d71f67b225f5fd3336f66c769bddf273d5611c33e
    m.SearchUrl.AddHeader("Content-Type","text/html")
    m.SearchUrl.AddHeader("Content-Length",ContentLength&.ToStr()) 'for example: 95948
    m.SearchUrl.AddHeader("x-amz-acl","public-read")
    m.SearchUrl.AddHeader("x-amz-date", HeaderDate) 'format 20190914T181425Z
    m.SearchUrl.Addheader("x-amz-storage-class", "STANDARD")
    m.SearchUrl.AddHeader("x-amz-content-sha256", Content_sha256) 'for example: ec7a120530d1e03657be8ef41ca89dd7450526eb1a59ede2be049044e39bfd2d
    m.SearchUrl.SetUrl("https://grouphiit-roku.s3.amazonaws.com")
    m.SearchUrl.RetainBodyOnError(true)
    Response = m.SearchUrl.AsyncPostFromFiletofile("tmp:/Payload.txt","tmp:/Response.txt")  'I'm not sure what the payload is supposed to be!!!!!
    ? "response.txt = ";readasciifile("tmp:/response.txt")

I'm not sure what the "payload" is supposed to be. If I make the paylolad equal to simply the file (for example "tmp:/Myfile.txt"), then I get a malformed XML error back from AWS S3 -- so I'm guessing AWS is expecting an XML string.

If cound this which indicates the AWS S3 XSD form for a PutObject is:

  <xsd:element name="PutObject">
    <xsd:complexType>
      <xsd:sequence>
        <xsd:element name="Bucket" type="xsd:string"/>
        <xsd:element name="Key" type="xsd:string"/>
        <xsd:element name="Metadata" type="tns:MetadataEntry" minOccurs="0" maxOccurs="100"/>
        <xsd:element name="ContentLength" type="xsd:long"/>
        <xsd:element name="AccessControlList" type="tns:AccessControlList" minOccurs="0"/>
        <xsd:element name="StorageClass" type="tns:StorageClass" minOccurs="0"/>
        <xsd:element name="AWSAccessKeyId" type="xsd:string" minOccurs="0"/>
        <xsd:element name="Timestamp" type="xsd:dateTime" minOccurs="0"/>
        <xsd:element name="Signature" type="xsd:string" minOccurs="0"/>
        <xsd:element name="Credential" type="xsd:string" minOccurs="0"/>
      </xsd:sequence>
    </xsd:complexType>
  </xsd:element>

If I send an XML string that has those elements (am I right that the ones with "minOccurs = 0" are optional?), I get a time out error and nothing happens.

Has anyone successfully uploaded to AWS using a PUT (or alternatively, using the AWS presigned URL method). If so, I'd greatly appreciate guidance on how to make this happen.

Thanks in advance for your help.

juanricos · Tue Sep 17, 2019 9:07 pm

I think what you may be missing is that the filename goes on the end of the URL. The "PUT /myfile.txt HTTP 1.1" is not a header and should not be added as one. Example:

   m.SearchUrl.SetRequest("PUT")
    m.SearchUrl.AddHeader("Host",Host)  'Host format = bucket-name.s3.amazonaws.com
    m.SearchUrl.AddHeader("Date", DateString)  'Date format = Sat, Sep 14, 2019 18:14:25 GMT
    m.SearchUrl.AddHeader("Authorization", AuthorizationHeaderString) 'for example: AWS4-HMAC-SHA256 Credential=AKIAxxxxxxxxxxxxxxxx/20190914/us-east-1/s3/aws4_request,SignedHeaders=date;content-length;content-type;host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-storage-class,Signature=3af54f13f05222daff50047d71f67b225f5fd3336f66c769bddf273d5611c33e
    m.SearchUrl.AddHeader("Content-Type","text/html")
    m.SearchUrl.AddHeader("Content-Length",ContentLength&.ToStr()) 'for example: 95948
    m.SearchUrl.AddHeader("x-amz-acl","public-read")
    m.SearchUrl.AddHeader("x-amz-date", HeaderDate) 'format 20190914T181425Z
    m.SearchUrl.Addheader("x-amz-storage-class", "STANDARD")
    m.SearchUrl.AddHeader("x-amz-content-sha256", Content_sha256) 'for example: ec7a120530d1e03657be8ef41ca89dd7450526eb1a59ede2be049044e39bfd2d
    m.SearchUrl.SetUrl("https://bucket-name.s3.amazonaws.com/myFile.txt")
    m.SearchUrl.RetainBodyOnError(true)
    Response = m.SearchUrl.AsyncPostFromFiletofile("tmp:/myFile.txt","tmp:/Response.txt")  'from file is the file you are sending
    ? "response.txt = ";readasciifile("tmp:/response.txt")

wshirley · Wed Sep 18, 2019 9:56 am

Thank you. That did the trick. Problem solved.

juanricos · Sun Sep 22, 2019 5:22 pm

OK, that's great, happy to help. The way AWS formatted those "examples" is confusing.

HTTP PUT to AWS S3 - Payload Issue

HTTP PUT to AWS S3 - Payload Issue

Re: HTTP PUT to AWS S3 - Payload Issue

Re: HTTP PUT to AWS S3 - Payload Issue

Re: HTTP PUT to AWS S3 - Payload Issue

Who is online