Upload
The upload resource represents data in the state of being uploaded. It is nested under the scope of the resource the data belongs to, that is under a clip or a file.
The following steps are necessary to perform an upload:
- Acquire the URI of the parent resource
- Create the resource (file or clip) you want to upload to
- Create the upload, passing the total size.
- Upload the first chunk using the update action.
- Repeat the last step until the entire data has been transferred
Chunking
Up to a certain file size, splitting the data in chunks is optional. However it is recommended to do so for large files. The chunk size is up to the client application, reasonable chunk sizes are a few megabytes up to 64 MB. The chunk size should not exceed 128 MB and there is a hard limit to the chunk size of 1000 MB. Larger files have to be split into chunks.
All chunks, except the last one, must have the same size. The last chunk must not be larger than the other chunks.
Chunk size is taken from the upload of chunk 1 which must be provided first. Alternatively you can provide an additional parameter total_chunks which allows us to determine the chunk size from any chunk. In that case the order of chunks does not matter and it is allowed to send multiple chunks in parallel.
When all chunks are provided the upload is finished automatically.
Upload Hosts
Chunk upload must not be done to the main webgate server. Instead it needs to go to an upload server. The create call returns a list of valid upload servers, that may be used for uploads. E.g.
{ "status": 201,
"status_message": "Created",
"info": "",
"data": { "flowuploader::upload": {
"current_size": 0,
"total_size": 13,
"hosts": ["keks.webgate.io", "praline.webgate.io", ...],
"url": "..." }
} }
Since this list can change, it is not recommended to hard code one of these servers.
(Note: the create call was restricted to an upload host as well in the past, but that restriction has been removed.)
It is allowed to upload different chunks to different upload hosts.
Uploads to the main webgate server (e.g. https://webgate.io/) are rejected.
Call Details
| Attributes | ||||
|---|---|---|---|---|
| total_size | Integer | 12345 | create | Total size of the upload in bytes. |
| chunk | Integer | 2 | update | Number of chunk provided. Defaults to 1 |
| total_chunks | Integer | 3 | update | Total number of chunks. |
| Actions | |||
|---|---|---|---|
| Create | POST | {resource_path}/upload | necessary data: {"total_size": total upload size in bytes (int not string)} |
| Update | PUT | {resource_path}/upload?chunk=13&total_chunks=16 | Header must contain: "Content-Type: application/octet-stream" |
Both actions respond with data providing the current_size and total_size in the namespace flowuploader::upload E.g.
{ "status": 201,
"status_message": "Created",
"info": "",
"data": { "flowuploader::upload": {
"current_size": 0,
"total_size": 13,
"hosts": ["keks.webgate.io", "praline.webgate.io", ...],
"url": "..." }
} }
Error Handling
When a chunk is processed correctly the webserver will send a response with status 200 (OK).
Errors
Errors, that can occur during uploads, include:
- 404 in response to: POST/PUT The resource you are uploading to has been deleted. It is suggested to terminate the upload in this case.
- 422 in response to: POST An upload has been created already.
- 422 in response to: PUT Unexpected data has been received, e.g. a chunk number larger than the expected chunk numbers or an unexpected amount of data. Or the upload has not been created yet. It does not make sense to repeat the request in this case.
- In case of other errors, e.g. a generic 500 or errors on the network level, it makes sense to retry the upload of a chunk, probably on a different upload host.