Apps commonly request users to upload files, images or other data from their computer, and store the user content in backend object storage such as Amazon S3. In addition, apps such as those that use our open-source File Picker often allow users to select files in cloud storage to import into their app as well.

Transferring files from users’ cloud storage to an app’s backend storage usually involves writing server-side logic to copy the file over. However, this process is error-prone, since transfers could be interrupted, and requires additional complexity to handle.

What if there is an easier way? In this blog post, we will discuss how to use the Kloudless File Picker to easily transfer user content from cloud storage into an app’s backend storage.

The old way: Writing code to transfer data

Let’s use Kloudless’ own Storage API to demonstrate how an application would copy files from a user’s cloud storage account to its own storage account. The example API request below copies avatar.jpg (File ID fL3Rlc3QucG5n) from a Dropbox account to a bucket in an Amazon S3 account (Account ID 77; Folder ID Fn2k36s0t) using the File Copy endpoint.

The API requests would be even more complex if the application had to implement it entirely on its own—it would involve downloading the file and streaming it to the backend storage repository via an upload or multi-part upload, depending on the size.

The File ID mentioned above can be obtained by prompting the user to select a file in their cloud storage account via the Kloudless File Picker as shown below.

The Kloudless File Picker is pretty handy for this purpose, but it would be even better if the application servers did not have to worry about data transfer at all! Fortunately, the File Picker also supports this capability.

The new way: asynchronous uploads via the File Picker

The File Picker includes additional options to automatically copy data to an app’s configured backend storage. The copy_to_upload_location instructs the File Picker to launch a Copy job immediately after the user selects the file they’d like to import. This is much easier to implement than writing the logic to transfer the data via HTTP requests .

Configuring a default upload location for user content

To configure an upload location for user content, head to the Kloudless developer portal’s File Picker section.

In the example below, we’ve added two upload locations. The first is to the public_files folder in a Google Drive account (Account ID 1234; Folder ID Fabcdefg), and the second is to a root folder in a Dropbox account (Account ID 5678; Folder ID root). We commonly see Amazon S3 and Azure Blob Storage upload locations, since they are more appropriate for an app’s backend storage (see our other blog post for why!).

Add the JS snippet below into your code, or update your existing File Picker configuration to include the upload_location_account and upload_location_folder attributes to initialize the File Picker. Check out a fully functional demo here.

upload_location_account and upload_location_folder are only required if you’ve configured multiple upload locations for the app. Multiple upload locations help route uploads differently depending on which user uses the app.

With the configuration above in place, open the File Picker, connect a cloud storage account, and choose the file you’d like to copy to the app’s storage. The File Picker waits for a background task to complete the upload prior to returning the newly uploaded file’s metadata.

Making it asynchronous

The background task above eventually completes, but takes an indeterminate amount of time. To improve the user experience, the File Picker also supports asynchronous uploads via the Kloudless Tasks API. Set the copy_to_upload_location attribute to async to trigger this behavior.

With the new configuration, the File Picker returns a Task ID instead, such as f61a03f8-5c56-45d8-b697-692c415dfd78. Use this with the Tasks API endpoint linked to above to retrieve the task’s status and the new file’s metadata.

Kloudless handles all the steps required behind the scenes to ensure the file transfer is successful. This includes shifting from regular uploads to multi-part uploads for large files, varying connection options and protocols depending on the upstream storage service, refreshing OAuth tokens, and handling errors and rate limits.

Sign up for a free account to try out the File Picker and our cloud storage API .