Multi part write

[shaileshkm]

Hi,

I'm implementing multi part upload using REST API.
I have it working with Content-Range headers etc.
The documentation suggests to check the HTTP response header 'Range' to confirm how much was actually written.
In my case, I'm taking care of doing write in multiple of 256KB, except for the last part.
In such case (write part in multiple of 256KB), can the response ever be less than what I ask to write?

Example:
Send Content-Range: bytes 262144-786431/*
Here the request is to upload 512KB from offset 256KB.
In this case, Is it possible for the server to respond with
case 1:
Range: bytes=262144-524287
or
case 2:
Range: bytes=262144-550000

case 1 is little easier to handle as reattempt can be made by just adjusting 256KB boundary in buffers.
I don't know if case 2 is even possible as the next request to make must have
Content-Range: bytes 550000-812144
These values are not in multiple of 256KB.

[coryan]

In such case (write part in multiple of 256KB), can the response ever be less than what I ask to write?

I feel like this was asked elsewhere. My understanding is that:

• The backend always writes in blocks that are multiples of 256KiB, but this is not guaranteed by the documentation anywhere.
• A PUT request almost always writes all the bytes you send before returning, but it can happen that it does not.

In this case, Is it possible for the server to respond with

case 2:
Range: bytes=262144-550000

The short answer is "I do not know". The longe answers are:

• There is no guarantee it cannot happen
• It is better to assume it can happen
• You need to start sending multiples of 256KiB but from byte 550000 from now on.
• You probably should go through the sequence of steps to resume an upload in this case.

[shaileshkm]

Thanks Carlos. Is it possible to request for clarification in documentation?
If I have to do it, can you send me link where I can ask for it?

For case 2 possibility, I was thinking of following solution

Write from a valid 256KB boundary.
If response was
Range: bytes=262144-550000

then do
Content-Range: bytes 524288-786431/*

Here my assumption is Storage server code does not take exception to rewriting.

[coryan]

Rewriting data works, but (a) you must make sure you send the same bytes, (b) this is undocumented behavior, it may change in the future.

You can use the "Send Feedback" button directly in the documentation:

https://cloud.google.com/storage/docs/performing-resumable-uploads

There is also an issue tracker:

https://issuetracker.google.com/components/187243