1Net::Amazon::S3(3) User Contributed Perl Documentation Net::Amazon::S3(3)
2
6 Net::Amazon::S3 - Use the Amazon S3 - Simple Storage Service
7
9 version 0.991
10
12 use Net::Amazon::S3;
13 use Net::Amazon::S3::Authorization::Basic;
14 use Net::Amazon::S3::Authorization::IAM;
15 my $aws_access_key_id = 'fill me in';
16 my $aws_secret_access_key = 'fill me in too';
17 my $s3 = Net::Amazon::S3->new (
19 authorization_context => Net::Amazon::S3::Authorization::Basic->new (
20 aws_access_key_id => $aws_access_key_id,
21 aws_secret_access_key => $aws_secret_access_key,
22 ),
23 retry => 1,
24 );
25 # or use an IAM role.
27 my $s3 = Net::Amazon::S3->new (
28 authorization_context => Net::Amazon::S3::Authorization::IAM->new (
29 aws_access_key_id => $aws_access_key_id,
30 aws_secret_access_key => $aws_secret_access_key,
31 ),
32 retry => 1,
33 );
34 # a bucket is a globally-unique directory
36 # list all buckets that i own
37 my $response = $s3->buckets;
38 foreach my $bucket ( @{ $response->{buckets} } ) {
39 print "You have a bucket: " . $bucket->bucket . "\n";
40 }
41 # create a new bucket
43 my $bucketname = 'acmes_photo_backups';
44 my $bucket = $s3->add_bucket( { bucket => $bucketname } )
45 or die $s3->err . ": " . $s3->errstr;
46 # or use an existing bucket
48 $bucket = $s3->bucket($bucketname);
49 # store a file in the bucket
51 $bucket->add_key_filename( '1.JPG', 'DSC06256.JPG',
52 { content_type => 'image/jpeg', },
53 ) or die $s3->err . ": " . $s3->errstr;
54 # store a value in the bucket
56 $bucket->add_key( 'reminder.txt', 'this is where my photos are backed up' )
57 or die $s3->err . ": " . $s3->errstr;
58 # list files in the bucket
60 $response = $bucket->list_all
61 or die $s3->err . ": " . $s3->errstr;
62 foreach my $key ( @{ $response->{keys} } ) {
63 my $key_name = $key->{key};
64 my $key_size = $key->{size};
65 print "Bucket contains key '$key_name' of size $key_size\n";
66 }
67 # fetch file from the bucket
69 $response = $bucket->get_key_filename( '1.JPG', 'GET', 'backup.jpg' )
70 or die $s3->err . ": " . $s3->errstr;
71 # fetch value from the bucket
73 $response = $bucket->get_key('reminder.txt')
74 or die $s3->err . ": " . $s3->errstr;
75 print "reminder.txt:\n";
76 print " content length: " . $response->{content_length} . "\n";
77 print " content type: " . $response->{content_type} . "\n";
78 print " etag: " . $response->{content_type} . "\n";
79 print " content: " . $response->{value} . "\n";
80 # delete keys
82 $bucket->delete_key('reminder.txt') or die $s3->err . ": " . $s3->errstr;
83 $bucket->delete_key('1.JPG') or die $s3->err . ": " . $s3->errstr;
84 # and finally delete the bucket
86 $bucket->delete_bucket or die $s3->err . ": " . $s3->errstr;
87
89 This module provides a Perlish interface to Amazon S3. From the
90 developer blurb: "Amazon S3 is storage for the Internet. It is designed
91 to make web-scale computing easier for developers. Amazon S3 provides a
92 simple web services interface that can be used to store and retrieve
93 any amount of data, at any time, from anywhere on the web. It gives any
94 developer access to the same highly scalable, reliable, fast,
95 inexpensive data storage infrastructure that Amazon uses to run its own
96 global network of web sites. The service aims to maximize benefits of
97 scale and to pass those benefits on to developers".
98 To find out more about S3, please visit: http://s3.amazonaws.com/
100 To use this module you will need to sign up to Amazon Web Services and
102 provide an "Access Key ID" and " Secret Access Key". If you use this
103 module, you will incurr costs as specified by Amazon. Please check the
104 costs. If you use this module with your Access Key ID and Secret Access
105 Key you must be responsible for these costs.
106 I highly recommend reading all about S3, but in a nutshell data is
108 stored in values. Values are referenced by keys, and keys are stored in
109 buckets. Bucket names are global.
110 Note: This is the legacy interface, please check out
112 Net::Amazon::S3::Client instead.
113 Development of this code happens here:
115 https://github.com/rustyconover/net-amazon-s3
116 Bucket names with dots, HTTPS, and Signature V4
118 At the moment Amazon S3 doesn't play well with HTTPS and virtual bucket
119 hosts if bucket name contains dots.
120 Due the current implementation of Signature V4 handling you should use
122 workaround consisting of usage of region hostnames
123 my $bucket_region = $global_s3->bucket ($bucket)->_head_region;
125 my $region_s3 = Net::Amazon:S3->new (
127 ...,
128 vendor => Net::Amazon::S3::Vendor::Amazon->new (
129 host => "s3-$bucket_region.amazonaws.com",
130 use_virtual_host => 0,
131 ),
132 );
133 my $bucket = $region_s3->bucket ($bucket);
135 And use bucket instance / region s3 connection.
137
139 new
140 Create a new S3 client object. Takes some arguments:
141 authorization_context
143 Class that provides authorization information.
144 See one of available implementations for more
146 Net::Amazon::S3::Authorization::Basic
148 Net::Amazon::S3::Authorization::IAM
149 vendor
150 Instance of Net::Amazon::S3::Vendor holding vendor specific
151 deviations.
152 S3 became widely used object storage protocol with many vendors
154 providing different feature sets and different compatibility level.
155 One common difference is bucket's HEAD request to determine its
157 region.
158 To maintain currently known differences along with any differencies
160 that may rise in feature it's better to hold vendor specification
161 in dedicated classes. This also allows users to build their own
162 fine-tuned vendor classes.
163 Net::Amazon::S3::Vendor::Amazon
165 Net::Amazon::S3::Vendor::Generic
166 aws_access_key_id
167 Deprecated.
168 When used it's used to create authorization context.
170 Use your Access Key ID as the value of the AWSAccessKeyId parameter
172 in requests you send to Amazon Web Services (when required). Your
173 Access Key ID identifies you as the party responsible for the
174 request.
175 aws_secret_access_key
177 Deprecated.
178 When used it's used to create authorization context.
180 Since your Access Key ID is not encrypted in requests to AWS, it
182 could be discovered and used by anyone. Services that are not free
183 require you to provide additional information, a request signature,
184 to verify that a request containing your unique Access Key ID could
185 only have come from you.
186 DO NOT INCLUDE THIS IN SCRIPTS OR APPLICATIONS YOU DISTRIBUTE.
188 YOU'LL BE SORRY
189 aws_session_token
191 Deprecated.
192 When used it's used to create authorization context.
194 If you are using temporary credentials provided by the AWS Security
196 Token Service, set the token here, and it will be added to the
197 request in order to authenticate it.
198 use_iam_role
200 Deprecated.
201 When used it's used to create authorization context.
203 If you'd like to use IAM provided temporary credentials, pass this
205 option with a true value.
206 secure
208 Deprecated.
209 Set this to 0 if you don't want to use SSL-encrypted connections
211 when talking to S3. Defaults to 1.
212 To use SSL-encrypted connections, LWP::Protocol::https is required.
214 See #vendor and Net::Amazon::S3::Vendor.
216 keep_alive_cache_size
218 Set this to 0 to disable Keep-Alives. Default is 10.
219 timeout
221 How many seconds should your script wait before bailing on a
222 request to S3? Defaults to 30.
223 retry
225 If this library should retry upon errors. This option is
226 recommended. This uses exponential backoff with retries after 1,
227 2, 4, 8, 16, 32 seconds, as recommended by Amazon. Defaults to off.
228 When retry is on, request will be automatically retried when one of
230 following HTTP statuses happens
231 408 - Request Timeout
233 500 - Internal Server Error
234 502 - Bad Gateway
235 503 - Service Unavailable
236 504 - Gateway Timeout
237 For more details see LWP::UserAgent::Determined.
239 host
241 Deprecated.
242 The S3 host endpoint to use. Defaults to 's3.amazonaws.com'. This
244 allows you to connect to any S3-compatible host.
245 See #vendor and Net::Amazon::S3::Vendor.
247 use_virtual_host
249 Deprecated.
250 Use the virtual host method ('bucketname.s3.amazonaws.com') instead
252 of specifying the bucket at the first part of the path. This is
253 particularly useful if you want to access buckets not located in
254 the US-Standard region (such as EU, Asia Pacific or South America).
255 See
256 <http://docs.aws.amazon.com/AmazonS3/latest/dev/VirtualHosting.html>
257 for the pros and cons.
258 See #vendor and Net::Amazon::S3::Vendor.
260 authorization_method
262 Deprecated.
263 Authorization implementation package name.
265 This library provides Net::Amazon::S3::Signature::V2 and
267 Net::Amazon::S3::Signature::V4
268 Default is Signature 4 if host is "s3.amazonaws.com", Signature 2
270 otherwise
271 See #vendor and Net::Amazon::S3::Vendor.
273 error_handler_class
275 Error handler class name (package name), see
276 Net::Amazon::S3::Error::Handler for more.
277 Default: Net::Amazon::S3::Error::Handler::Legacy
279 error_handler
281 Instance of error handler class.
282 Notes
284 When using Net::Amazon::S3 in child processes using fork (such as in
286 combination with the excellent Parallel::ForkManager) you should create
287 the S3 object in each child, use a fresh LWP::UserAgent in each child,
288 or disable the LWP::ConnCache in the parent:
289 $s3->ua( LWP::UserAgent->new(
291 keep_alive => 0, requests_redirectable => [qw'GET HEAD DELETE PUT POST'] );
292 buckets
294 Returns undef on error, else hashref of results
295 add_bucket
297 # Create new bucket with default location
298 my $bucket = $s3->add_bucket ('new-bucket');
299 # Create new bucket in another location
301 my $bucket = $s3->add_bucket ('new-bucket', location_constraint => 'eu-west-1');
302 my $bucket = $s3->add_bucket ('new-bucket', { location_constraint => 'eu-west-1' });
303 my $bucket = $s3->add_bucket (bucket => 'new-bucket', location_constraint => 'eu-west-1');
304 my $bucket = $s3->add_bucket ({ bucket => 'new-bucket', location_constraint => 'eu-west-1' });
305 Method creates and returns new bucket.
307 In case of error it reports it and returns "undef" (refer "ERROR
309 HANDLING").
310 Recognized positional arguments (refer "CALLING CONVENTION")
312 bucket
314 Required, recognized as positional.
315 The name of the bucket you want to add.
317 Recognized optional arguments
319 acl
321 acl => 'private'
322 acl => Net::Amazon::S3::ACL::Canned->PRIVATE
323 acl => Net::Amazon::S3::ACL::Set->grant_read (email => 'foo@bar.baz')
324 Available since v0.94
326 Set ACL to the newly created bucket. Refer Net::Amazon::S3::ACL for
328 possibilities.
329 acl_short (deprecated)
331 Deprecated since v0.94
332 When specified its value is used to populate "acl" argument (unless
334 it exists).
335 location_constraint
337 Optional.
338 Sets the location constraint of the new bucket. If left
340 unspecified, the default S3 datacenter location will be used.
341 This library recognizes regions according Amazon S3 documentation
343 → <https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region>
345 → <https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateBucket.html#API_CreateBucket_RequestSyntax>
347 Provides operation CreateBucket
349 <https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateBucket.html>.
350 bucket BUCKET
352 # build bucket with guessed region
353 $s3->bucket ('foo');
354 $s3->bucket (bucket => 'foo');
355 $s3->bucket (name => 'foo');
356 # build with explicit region
358 $s3->bucket ('foo', region => 'bar');
359 Returns an (unverified) bucket object from an account. Does no network
361 access.
362 However, when guessing region, "HeadRegion" operation may be called
364 before first network access.
365 Region is mandatory when using Signature V4 authorization, which is
367 default for AWS. AWS limits number of HTTP requests, see
368 <https://aws.amazon.com/premiumsupport/knowledge-center/s3-request-limit-avoid-throttling/>
369 delete_bucket
371 $s3->delete_bucket ($bucket);
372 $s3->delete_bucket (bucket => $bucket);
373 Deletes bucket from account.
375 Returns "true" if the bucket is successfully deleted.
377 Returns "false" and reports an error otherwise (refer "ERROR HANDLING")
379 Positional arguments (refer "CALLING CONVENTION")
381 bucket
383 Required.
384 The name of the bucket or Net::Amazon::S3::Bucket instance you want
386 to delete.
387 Provides operation "DeleteBucket"
389 <https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucket.html>
390 list_bucket
392 List all keys in this bucket.
393 Takes a hashref of arguments:
395 MANDATORY
397 bucket
399 The name of the bucket you want to list keys on
400 OPTIONAL
402 prefix
404 Restricts the response to only contain results that begin with the
405 specified prefix. If you omit this optional argument, the value of
406 prefix for your query will be the empty string. In other words, the
407 results will be not be restricted by prefix.
408 delimiter
410 If this optional, Unicode string parameter is included with your
411 request, then keys that contain the same string between the prefix
412 and the first occurrence of the delimiter will be rolled up into a
413 single result element in the CommonPrefixes collection. These
414 rolled-up keys are not returned elsewhere in the response. For
415 example, with prefix="USA/" and delimiter="/", the matching keys
416 "USA/Oregon/Salem" and "USA/Oregon/Portland" would be summarized in
417 the response as a single "USA/Oregon" element in the CommonPrefixes
418 collection. If an otherwise matching key does not contain the
419 delimiter after the prefix, it appears in the Contents collection.
420 Each element in the CommonPrefixes collection counts as one against
422 the MaxKeys limit. The rolled-up keys represented by each
423 CommonPrefixes element do not. If the Delimiter parameter is not
424 present in your request, keys in the result set will not be rolled-
425 up and neither the CommonPrefixes collection nor the NextMarker
426 element will be present in the response.
427 max-keys
429 This optional argument limits the number of results returned in
430 response to your query. Amazon S3 will return no more than this
431 number of results, but possibly less. Even if max-keys is not
432 specified, Amazon S3 will limit the number of results in the
433 response. Check the IsTruncated flag to see if your results are
434 incomplete. If so, use the Marker parameter to request the next
435 page of results. For the purpose of counting max-keys, a 'result'
436 is either a key in the 'Contents' collection, or a delimited prefix
437 in the 'CommonPrefixes' collection. So for delimiter requests, max-
438 keys limits the total number of list results, not just the number
439 of keys.
440 marker
442 This optional parameter enables pagination of large result sets.
443 "marker" specifies where in the result set to resume listing. It
444 restricts the response to only contain results that occur
445 alphabetically after the value of marker. To retrieve the next page
446 of results, use the last key from the current page of results as
447 the marker in your next request.
448 See also "next_marker", below.
450 If "marker" is omitted,the first page of results is returned.
452 Returns undef on error and a hashref of data on success:
454 The hashref looks like this:
456 {
458 bucket => $bucket_name,
459 prefix => $bucket_prefix,
460 common_prefixes => [$prefix1,$prefix2,...]
461 marker => $bucket_marker,
462 next_marker => $bucket_next_available_marker,
463 max_keys => $bucket_max_keys,
464 is_truncated => $bucket_is_truncated_boolean
465 keys => [$key1,$key2,...]
466 }
467 Explanation of bits of that:
469 common_prefixes
471 If list_bucket was requested with a delimiter, common_prefixes will
472 contain a list of prefixes matching that delimiter. Drill down
473 into these prefixes by making another request with the prefix
474 parameter.
475 is_truncated
477 B flag that indicates whether or not all results of your query were
478 returned in this response. If your results were truncated, you can
479 make a follow-up paginated request using the Marker parameter to
480 retrieve the rest of the results.
481 next_marker
483 A convenience element, useful when paginating with delimiters. The
484 value of "next_marker", if present, is the largest (alphabetically)
485 of all key names and all CommonPrefixes prefixes in the response.
486 If the "is_truncated" flag is set, request the next page of results
487 by setting "marker" to the value of "next_marker". This element is
488 only present in the response if the "delimiter" parameter was sent
489 with the request.
490 Each key is a hashref that looks like this:
492 {
494 key => $key,
495 last_modified => $last_mod_date,
496 etag => $etag, # An MD5 sum of the stored content.
497 size => $size, # Bytes
498 storage_class => $storage_class # Doc?
499 owner_id => $owner_id,
500 owner_displayname => $owner_name
501 }
502 list_bucket_all
504 List all keys in this bucket without having to worry about 'marker'.
505 This is a convenience method, but may make multiple requests to S3
506 under the hood.
507 Takes the same arguments as list_bucket.
509 _perform_operation
511 my $response = $s3->_perform_operation ('Operation' => (
512 # ... operation request parameters
513 ));
514 Internal operation implementation method, takes request construction
516 parameters, performs necessary HTTP requests(s) and returns Response
517 instance.
518 Method takes same named parameters as realted Request class.
520 Method provides available contextual parameters by default (eg s3,
522 bucket)
523 Method invokes contextual error handler.
525
527 Available since v0.97 - calling convention extentend
528 In order to make method calls somehow consistent, backward compatible,
530 and extendable, API's methods support multiple ways how to provide
531 their arguments
532 plain named arguments (preferred)
534 method (named => 'argument', another => 'argument');
535 trailing configuration hash
537 method ({ named => 'argument', another => 'argument' });
538 method (positional, { named => 'argument', another => 'argument' } );
539 Last argument of every method can be configuration hash, treated as
541 additional named arguments. Can be combined with named arguments.
542 positional arguments with optional named arguments
544 method (positional, named => 'argument', another => 'argument');
545 method (positional, { named => 'argument', another => 'argument' } );
546 For methods supporting mandatory positional arguments additional
548 named arguments and/or configuration hash is supported.
549 Named arguments or configuration hash can specify value of
551 positional arguments as well removing it from list of required
552 positional arguments for given call (see example)
553 $s3->bucket->add_key ('key', 'value', acl => $acl);
555 $s3->bucket->add_key ('value', key => 'key', acl => $acl);
556 $s3->bucket->add_key (key => 'key', value => 'value', acl => $acl);
557
559 Net::Amazon::S3 supports pluggable error handling via
560 Net::Amazon::S3::Error::Handler.
561 When response ends up with an error, every method reports it, and in
563 case it receives control back (no exception), it returns "undef".
564 Default error handling for Net::Amazon::S3 is
566 Net::Amazon::S3::Error::Handler::Legacy which (mostly) sets "err" and
567 "errstr".
568
570 This module contains code modified from Amazon that contains the
571 following notice:
572 # This software code is made available "AS IS" without warranties of any
574 # kind. You may copy, display, modify and redistribute the software
575 # code either by itself or as incorporated into your code; provided that
576 # you do not remove any proprietary notices. Your use of this software
577 # code is at your own risk and you waive any claim against Amazon
578 # Digital Services, Inc. or its affiliates with respect to your use of
579 # this software code. (c) 2006 Amazon Digital Services, Inc. or its
580 # affiliates.
581
583 Testing S3 is a tricky thing. Amazon wants to charge you a bit of money
584 each time you use their service. And yes, testing counts as using.
585 Because of this, the application's test suite skips anything
586 approaching a real test unless you set these three environment
587 variables:
588 AMAZON_S3_EXPENSIVE_TESTS
590 Doesn't matter what you set it to. Just has to be set
591 AWS_ACCESS_KEY_ID
593 Your AWS access key
594 AWS_ACCESS_KEY_SECRET
596 Your AWS sekkr1t passkey. Be forewarned that setting this
597 environment variable on a shared system might leak that information
598 to another user. Be careful.
599
601 Leon Brocard <acme@astray.com> and unknown Amazon Digital Services
602 programmers.
603 Brad Fitzpatrick <brad@danga.com> - return values, Bucket object
605 Pedro Figueiredo <me@pedrofigueiredo.org> - since 0.54
607 Branislav Zahradník <barney@cpan.org> - since v0.81
609
611 Net::Amazon::S3::Bucket
612
614 Branislav Zahradník <barney@cpan.org>
615
617 This software is copyright (c) 2022 by Amazon Digital Services, Leon
618 Brocard, Brad Fitzpatrick, Pedro Figueiredo, Rusty Conover, Branislav
619 Zahradník.
620 This is free software; you can redistribute it and/or modify it under
622 the same terms as the Perl 5 programming language system itself.
623perl v5.36.0 2023-03-22 Net::Amazon::S3(3)