Object的分块上传
更新时间：2022-10-21
除了通过putObject()方法上传文件到BOS以外，BOS还提供了另外一种上传模式：分块上传（Multipart Upload）。用户可以在如下的应用场景内（但不仅限于此），使用分块上传模式，如：
- 需要支持断点上传。
- 上传超过5GB大小的文件。
- 网络条件较差，和BOS的服务器之间的连接经常断开。
- 需要流式地上传文件。
- 上传文件之前，无法确定上传文件的大小。
分块完成Multipart Upload
假设有一个文件，本地路径为/path/to/file.zip，由于文件比较大，使用分块上传其传输到BOS中。基本流程：
- 初始化Multipart Upload。
- 上传分块。
- 完成分块上传。
初始化Multipart Upload
使用initiateMultipartUpload方法来初始化一个分块上传事件：
示例代码：
Swift
1BOSInitiateMultipartUploadRequest* initMPRequest = [[BOSInitiateMultipartUploadRequest alloc] init];
2initMPRequest.bucket = @"<bucketname>";
3initMPRequest.key = @"<objectname>";
4initMPRequest.contentType = @"<content type>";
5
6__block BOSInitiateMultipartUploadResponse* initMPResponse = nil;
7BCETask* task = [client initiateMultipartUpload:initMPRequest];
8task.then(^(BCEOutput* output) {
9 if (output.response) {
10 initMPResponse = (BOSInitiateMultipartUploadResponse*)output.response;
11 NSLog(@"initiate multipart upload success!");
12 }
13
14 if (output.error) {
15 NSLog(@"initiate multipart upload failure");
16 }
17});
18[task waitUtilFinished];
19
20NSString* uploadID = initMPResponse.uploadId;说明：
initiateMultipartUpload的返回结果中含有uploadId，它是区分分块上传事件的唯一标识，在后面的操作中，我们将用到它。
上传分块
将文件分块上传。
示例代码
Swift
1// 计算分块个数
2NSString* file = @"/path/to/file.zip";
3NSDictionary<NSString*, id>* attr = [[NSFileManager defaultManager] attributesOfItemAtPath:file error:nil];
4uint64_t fileSize = attr.fileSize;
5uint64_t partSize = 1024 * 1024 * 5L;
6uint64_t partCount = fileSize / partSize;
7if (fileSize % partSize != 0) {
8 ++partCount;
9}
10
11NSMutableArray<BOSPart*>* parts = [NSMutableArray array];
12NSFileHandle* handle = [NSFileHandle fileHandleForReadingAtPath:@"/path/to/file.zip"];
13for (uint64_t i = 0; i < partCount; ++i) {
14 // seek
15 uint64_t skip = partSize * i;
16 [handle seekToFileOffset:skip];
17 uint64_t size = (partSize < fileSize - skip) ? partSize : fileSize - skip;
18
19 // data
20 NSData* data = [handle readDataOfLength:size];
21
22 // request
23 BOSUploadPartRequest* uploadPartRequest = [[BOSUploadPartRequest alloc] init];
24 uploadPartRequest.bucket = @"<bucketname>";
25 uploadPartRequest.key = @"<objectname>";
26 uploadPartRequest.objectData.data = data;
27 uploadPartRequest.partNumber = i + 1;
28 uploadPartRequest.uploadId = uploadID;
29
30 __block BOSUploadPartResponse* uploadPartResponse = nil;
31 task = [client uploadPart:uploadPartRequest];
32 task.then(^(BCEOutput* output) {
33 if (output.response) {
34 uploadPartResponse = (BOSUploadPartResponse*)output.response;
35 BOSPart* part = [[BOSPart alloc] init];
36 part.partNumber = i + 1;
37 part.eTag = uploadPartResponse.eTag;
38 [parts addObject:part];
39 }
40 });
41 [task waitUtilFinished];
42}注意：上面代码的核心是调用
uploadPart方法来上传每一个分块，但是要注意以下几点：
- uploadPart方法要求除最后一个Part以外，其他的Part大小都要大于等于5MB。但是Upload Part接口并不会立即校验上传Part的大小；只有当Complete Multipart Upload的时候才会校验。
- 为了保证数据在网络传输过程中不出现错误，建议您在
uploadPart后，使用每个分块BOS返回的Content-MD5值分别验证已上传分块数据的正确性。当所有分块数据合成一个Object后，不再含MD5值。- Part号码的范围是1~10000。如果超出这个范围，BOS将返回InvalidArgument的错误码。
- 每次上传Part时都要把流定位到此次上传块开头所对应的位置。
- 每次上传Part之后，BOS的返回结果会包含一个
BOSPart对象，它是上传块的ETag与块编号（PartNumber）的组合，在后续完成分块上传的步骤中会用到它，因此需要将其保存起来。一般来讲这些BOSPart对象将被保存到数组中。
完成分块上传
示例代码
Swift
1BOSCompleteMultipartUploadRequest* compMultipartRequest = [[BOSCompleteMultipartUploadRequest alloc] init];
2compMultipartRequest.bucket = @"<bucketname>";
3compMultipartRequest.key = @"<objectname>";
4compMultipartRequest.uploadId = uploadID;
5compMultipartRequest.parts = parts;
6
7__block BOSCompleteMultipartUploadResponse* complResponse = nil;
8task = [client completeMultipartUpload:compMultipartRequest];
9task.then(^(BCEOutput* output) {
10 if (output.response) {
11 complResponse = (BOSCompleteMultipartUploadResponse*)output.response;
12 NSLog(@"complte multiparts success!");
13 }
14
15 if (output.error) {
16 NSLog(@"complte multiparts failure %@", output.error);
17 }
18});
19[task waitUtilFinished];说明：上面代码中的
parts是第二步中保存的parts的列表，BOS收到用户提交的Part列表后，会逐一验证每个数据Part的有效性。当所有的数据Part验证通过后，BOS将把这些数据part组合成一个完整的Object。
完整示例
Swift
1#import <BaiduBCEBasic/BaiduBCEBasic.h>
2#import <BaiduBCEBOS/BaiduBCEBOS.h>
3
4void example(void) {
5// 初始化
6BCECredentials* credentials = [[BCECredentials alloc] init];
7credentials.accessKey = @"<access key>";
8credentials.secretKey = @"<secret key>";
9BOSClientConfiguration* configuration = [[BOSClientConfiguration alloc] init];
10configuration.credentials = credentials;
11
12BOSClient* client = [[BOSClient alloc] initWithConfiguration:configuration];
13
14// 初始化分块上传
15BOSInitiateMultipartUploadRequest* initMPRequest = [[BOSInitiateMultipartUploadRequest alloc] init];
16initMPRequest.bucket = @"<bucketname>";
17initMPRequest.key = @"<objectname>";
18initMPRequest.contentType = @"<content type>";
19
20 __block BOSInitiateMultipartUploadResponse* initMPResponse = nil;
21 BCETask* task = [client initiateMultipartUpload:initMPRequest];
22 task.then(^(BCEOutput* output) {
23 if (output.response) {
24 initMPResponse = (BOSInitiateMultipartUploadResponse*)output.response;
25 NSLog(@"initiate multipart upload success!");
26 }
27
28 if (output.error) {
29 NSLog(@"initiate multipart upload failure");
30 }
31 });
32 [task waitUtilFinished];
33
34 NSString* uploadID = initMPResponse.uploadId;
35
36 // 计算分块个数
37 NSString* file = @"/path/to/file.zip";
38 NSDictionary<NSString*, id>* attr = [[NSFileManager defaultManager] attributesOfItemAtPath:file error:nil];
39 uint64_t fileSize = attr.fileSize;
40 uint64_t partSize = 1024 * 1024 * 5L;
41 uint64_t partCount = fileSize / partSize;
42 if (fileSize % partSize != 0) {
43 ++partCount;
44 }
45
46 NSMutableArray<BOSPart*>* parts = [NSMutableArray array];
47
48 NSFileHandle* handle = [NSFileHandle fileHandleForReadingAtPath:@"/path/to/file.zip"];
49 for (uint64_t i = 0; i < partCount; ++i) {
50 // seek
51 uint64_t skip = partSize * i;
52 [handle seekToFileOffset:skip];
53 uint64_t size = (partSize < fileSize - skip) ? partSize : fileSize - skip;
54
55 // data
56 NSData* data = [handle readDataOfLength:size];
57
58 // request
59 BOSUploadPartRequest* uploadPartRequest = [[BOSUploadPartRequest alloc] init];
60 uploadPartRequest.bucket = @"<bucketname>";
61 uploadPartRequest.key = @"<objectname>";
62 uploadPartRequest.objectData.data = data;
63 uploadPartRequest.partNumber = i + 1;
64 uploadPartRequest.uploadId = uploadID;
65
66 __block BOSUploadPartResponse* uploadPartResponse = nil;
67 task = [client uploadPart:uploadPartRequest];
68 task.then(^(BCEOutput* output) {
69 if (output.response) {
70 uploadPartResponse = (BOSUploadPartResponse*)output.response;
71 BOSPart* part = [[BOSPart alloc] init];
72 part.partNumber = i + 1;
73 part.eTag = uploadPartResponse.eTag;
74 [parts addObject:part];
75 }
76 });
77 [task waitUtilFinished];
78 }
79
80 BOSCompleteMultipartUploadRequest* compMultipartRequest = [[BOSCompleteMultipartUploadRequest alloc] init];
81 compMultipartRequest.bucket = @"<bucketname>";
82 compMultipartRequest.key = @"<objectname>";
83 compMultipartRequest.uploadId = uploadID;
84 compMultipartRequest.parts = parts;
85
86 __block BOSCompleteMultipartUploadResponse* complResponse = nil;
87 task = [client completeMultipartUpload:compMultipartRequest];
88 task.then(^(BCEOutput* output) {
89 if (output.response) {
90 complResponse = (BOSCompleteMultipartUploadResponse*)output.response;
91 NSLog(@"complte multiparts success!");
92 }
93
94 if (output.error) {
95 NSLog(@"complte multiparts failure %@", output.error);
96 }
97 });
98 [task waitUtilFinished];
99}取消分块上传
用户可以使用abortMultipartUpload方法取消分块上传。
示例代码：
Swift
1BOSAbortMultipartUploadRequest* abortRequest = [[BOSAbortMultipartUploadRequest alloc] init];
2abortRequest.bucket = @"bucket";
3abortRequest.key = @"<objectname>";
4abortRequest.uploadId = uploadID;
5
6__block BOSAbortMultipartUploadResponse* abortResponse = nil;
7task = [client abortMultipartUpload:abortRequest];
8task.then(^(BCEOutput* output) {
9 if (output.response) {
10 abortResponse = (BOSAbortMultipartUploadResponse*)output.response;
11 NSLog(@"abort multiparts success!");
12 }
13
14 if (output.error) {
15 NSLog(@"abort multiparts failure %@", output.error);
16 }
17});
18[task waitUtilFinished];获取未完成的分块上传
用户可以使用listMultipartUploads方法获取Bucket内未完成的分块上传事件。
基本流程
- 创建BOSListMultipartUploadsRequest类的实例,传入
<BucketName>参数。 - 创建BOSClient类的实例,执行BOSClient listMultipartUploads方法。
- listMultipartUploads返回所有未完成的分块上传信息。
示例代码
Swift
1BOSListMultipartUploadsRequest* listMultipartRequest = [[BOSListMultipartUploadsRequest alloc] init];
2listMultipartRequest.bucket = @"<bucketname>";
3
4__block BOSListMultipartUploadsResponse* listMultipartResponse = nil;
5task = [client listMultipartUploads:listMultipartRequest];
6task.then(^(BCEOutput* output) {
7 if (output.response) {
8 listMultipartResponse = (BOSListMultipartUploadsResponse*)output.response;
9 NSLog(@"list multipart success");
10 }
11
12 if (output.error) {
13 NSLog(@"list multipart failure %@", output.error);
14 }
15});
16[task waitUtilFinished];注意：
- 默认情况下，如果Bucket中的分块上传事件的数目大于1000，则只会返回1000个Object，并且返回结果中IsTruncated的值为True，同时返回nextKeyMarker作为下次读取的起点。
- 若想获取更多分块上传事件，可以使用keyMarker参数分次读取。
完整示例
Swift
1#import <BaiduBCEBasic/BaiduBCEBasic.h>
2#import <BaiduBCEBOS/BaiduBCEBOS.h>
3
4void example(void) {
5 // 初始化
6 BCECredentials* credentials = [[BCECredentials alloc] init];
7 credentials.accessKey = @"<access key>";
8 credentials.secretKey = @"<secret key>";
9 BOSClientConfiguration* configuration = [[BOSClientConfiguration alloc] init];
10 configuration.credentials = credentials;
11
12 BOSClient* client = [[BOSClient alloc] initWithConfiguration:configuration];
13
14 BOSListMultipartUploadsRequest* listMultipartRequest = [[BOSListMultipartUploadsRequest alloc] init];
15 listMultipartRequest.bucket = @"<bucketname>";
16
17 __block BOSListMultipartUploadsResponse* listMultipartResponse = nil;
18 BCETask* task = [client listMultipartUploads:listMultipartRequest];
19 task.then(^(BCEOutput* output) {
20 if (output.response) {
21 listMultipartResponse = (BOSListMultipartUploadsResponse*)output.response;
22 NSLog(@"list multipart success");
23 }
24
25 if (output.error) {
26 NSLog(@"list multipart failure %@", output.error);
27 }
28 });
29 [task waitUtilFinished];
30
31 for (BOSMultipartUpload* upload in listMultipartResponse.uploads) {
32 NSLog(@"upload id : %@", upload.uploadId);
33 }
34}获取所有已上传的分块信息
用户可以使用listParts方法获取某个上传事件中所有已上传的块。
基本流程
- 创建BOSListPartsRequest类的实例,传入
<BucketName>,<ObjectKey>,<UploadId>参数。 - 创建BOSClient类的实例,执行BOSClient listParts方法。
- listParts返回所有已上传part的信息。
示例代码
Swift
1BOSListPartsRequest* listPartsRequest = [[BOSListPartsRequest alloc] init];
2listPartsRequest.bucket = @"<bucketname>";
3listPartsRequest.key = @"<objectname>";
4listPartsRequest.uploadId = @"<upload id>";;
5
6__block BOSListPartsResponse* listPartsResponse = nil;
7BCETask* task = [client listParts:listPartsRequest];
8task.then(^(BCEOutput* output) {
9 if (output.response) {
10 listPartsResponse = (BOSListPartsResponse*)output.response;
11 NSLog(@"list parts success!");
12 }
13
14 if (output.error) {
15 NSLog(@"list part failure %@", output.error);
16 }
17});
18[task waitUtilFinished];
19
20for (BOSPart* part in listPartsResponse.parts) {
21 NSLog(@"part etag %@", part.eTag);
22}注意：
- 默认情况下，如果Bucket中的分块上传事件的数目大于1000，则只会返回1000个Object，并且返回结果中IsTruncated的值为True，同时返回NextPartNumberMarker作为下次读取的起点。
- 若想获取更多已上传的分块信息，可以使用PartNumberMarker参数分次读取。
完整示例
Swift
1#import <BaiduBCEBasic/BaiduBCEBasic.h>
2#import <BaiduBCEBOS/BaiduBCEBOS.h>
3
4void example(void) {
5 // 初始化
6 BCECredentials* credentials = [[BCECredentials alloc] init];
7 credentials.accessKey = @"<access key>";
8 credentials.secretKey = @"<secret key>";
9 BOSClientConfiguration* configuration = [[BOSClientConfiguration alloc] init];
10 configuration.credentials = credentials;
11
12 BOSClient* client = [[BOSClient alloc] initWithConfiguration:configuration];
13
14 BOSListPartsRequest* listPartsRequest = [[BOSListPartsRequest alloc] init];
15 listPartsRequest.bucket = @"<bucketname>";
16 listPartsRequest.key = @"<objectname>";
17 listPartsRequest.uploadId = @"<upload id>";;
18
19 __block BOSListPartsResponse* listPartsResponse = nil;
20 BCETask* task = [client listParts:listPartsRequest];
21 task.then(^(BCEOutput* output) {
22 if (output.response) {
23 listPartsResponse = (BOSListPartsResponse*)output.response;
24 NSLog(@"list parts success!");
25 }
26
27 if (output.error) {
28 NSLog(@"list part failure %@", output.error);
29 }
30 });
31 [task waitUtilFinished];
32
33 for (BOSPart* part in listPartsResponse.parts) {
34 NSLog(@"part etag %@", part.eTag);
35 }
36}