platformOS Community

tommyooc Aug 26 2021 at 01:19

I am trying to upload a binary file / remote file using api_send_call mutation. The flow is similar to cURL and file_get_content in PHP.

Here the code:

{%- liquid
    assign response = "{}" | parse_json
    hash_assign response['remote_file'] = context.params.file_url | split: "?" | first
    hash_assign response['filename'] = response.remote_file | split: "/" | last
    hash_assign response['filepath'] = "modules/homepage/" | append: response.filename
    hash_assign response['filetype'] = 'image/png'
-%}

{%- graphql get_assets_presign_url, path: response.filepath -%}
mutation( $path: String! ) {
    ooc: admin_assets_presign_urls(
        paths: {
            path: $path
        }
    ) {
        rs: urls {
            access_url
            path
            upload_url
        }
    }
}
{%- endgraphql -%}

{%- hash_assign response['assets_payload'] = get_assets_presign_url | dig: "ooc", "rs", 0 -%}

{% background priority: 'high', source_name: "PUT Method Upload", data: response %}
    {%- graphql PUTOBJECT, payload: data -%}
        mutation ApiCallSend( $payload: HashObject ) {
            ooc: api_call_send(
                template: {
                    name: "modules/homepage/putobject"
                }
                data: $payload
            ) {
                rs: response {
                    body
                    status
                    headers
                }
            }
        }
    {%- endgraphql -%}
    {%- log "----------- BACKGROUND JOB: PUT Method Upload ---------------", env: "staging" -%}
    {%- log PUTOBJECT, type: "Upload remote file", env: "staging" -%}
{%- endbackground -%}

And this is the api_call code:

---
to: '{{ form.assets_payload.upload_url }}'
format: http
delay: 0
request_type: PUT
request_headers: '{
    "Content-Type": "{{ form.filetype }}"
}'
---
{{- form.remote_file | download_file | html_safe -}}

I'm using PUT method.

But, I always get an "Access Denied" error message when I use:

to: '{{ form.assets_payload.upload_url }}'

However the upload is successful, if I use the Presigned URL directly like this:

to: 'https://HOST&REGION.amazonaws.com/uploads.staging.oregon.platform-os.com/instances/INSTANCE NUMBER/assets/modules/homepage/cow.png?x-amz-acl=public-read&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=CREDENTIAL%2F20210825%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20210825T230934Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Signature=SIGNATURE3aaa3f6b8211b40ad8feba963c987c1'

Even though it worked, I got a Graphql Error message: "string contains null bytes url"...

And I found another way to upload a binary file, from this page: https://documentation.platformos.com/developer-guide/apis/sending-binary-file-using-api-call-notification, but I don't know how to make it work with amazon S3.

Thanks.

Pawel Aug 26 2021 at 09:46

If using URL from form.assets_payload.upload_url doesn't work while hardcoding does work, my first step would be to {% log %} the first one and compare - it shouldn't matter how the URL gets there, so I would guess its one of two things:

upload_url is not there
url encoding comes into play

Error message suggests the first one.

I would also note that presign URL usually has a placeholder for the filename which is populated by filename being sent in, and in your hardcoded URL it is already filled in, that might also matter somehow, apart from overriding the same file every test.

Having said that, I think i would first try my luck with documented way of doing it which you mentioned later in the post.

Using https://documentation.platformos.com/developer-guide/apis/sending-binary-file-using-api-call-notification should was developed exactly for cases like yours, so i hope it works like this, putting your example data in it:

---
name: send_file
to: '{{ form.assets_payload.upload_url }}'
format: http
request_type: PUT
---
{
  "file": {
    "url": "{{ form.remote_file }}",
    "content_type": "{{ form.filetype }}"
  }
}

tommyooc Aug 27 2021 at 01:17
Hi Pawel. Thanks for the answer. I just tried with your suggestion. But it doesn't work, I get the same error message, "Access denied". I used a hardcoded URL and it worked, since the expiry date is 900, I don't think it matters, in this case I mean to make sure if uploading using api_call_send is possible. { "ooc": { "rs": { "body": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>XTFCD4WY24CNKC7Q</RequestId><HostId>Rx5CrG8a8TECxBBKWmhcDKZqcnwM4/tg8bhsaA2Bm4A09yafJqNNTmB26dtG22nfLHm/9Ihb1G8=</HostId></Error>", "status": 403, "headers": { "x-amz-request-id": [ "XTFCD4WY24CNKC7Q" ], "x-amz-id-2": [ "Rx5CrG8a8TECxBBKWmhcDKZqcnwM4/tg8bhsaA2Bm4A09yafJqNNTmB26dtG22nfLHm/9Ihb1G8=" ], "content-type": [ "application/xml" ], "transfer-encoding": [ "chunked" ], "date": [ "Fri, 27 Aug 2021 01:07:39 GMT" ], "server": [ "AmazonS3" ], "connection": [ "close" ] } } } }

tommyooc Aug 27 2021 at 05:37

Hi @pawel, you are correct,,,, the upload URL is encoded. Here it's the updated api_call_send

---
name: putobject
to: "{{- form.assets_presign.upload_url | html_safe -}}"
delay: 10
format: http
request_type: "PUT"
request_headers: '
    {
        "Cache-Control": "public, max-age=900",
        "Content-Type": "{{- form.filetype -}}"
    }
'
callback: >
    {% log form.assets_presign.upload_url, type: 'Upload URL' %}
---
{{ form.remote_file | download_file | html_safe }}

But even if the upload is successful, I still get the graphql error message: "GraphqlTagError: string contains null bytes"

Cheers!

How to upload a binary file to S3 using api_send_call mutation, is this possible?