1s3cmd(1) General Commands Manual s3cmd(1)
4
8 s3cmd - tool for managing Amazon S3 storage space and Amazon CloudFront
9 content delivery network
10
12 s3cmd [OPTIONS] COMMAND [PARAMETERS]
13
15 s3cmd is a command line client for copying files to/from Amazon S3
16 (Simple Storage Service) and performing other related tasks, for in‐
17 stance creating and removing buckets, listing objects, etc.
18
21 s3cmd can do several actions specified by the following commands.
22 s3cmd mb s3://BUCKET
24 Make bucket
25 s3cmd rb s3://BUCKET
27 Remove bucket
28 s3cmd ls [s3://BUCKET[/PREFIX]]
30 List objects or buckets
31 s3cmd la
33 List all object in all buckets
34 s3cmd put FILE [FILE...] s3://BUCKET[/PREFIX]
36 Put file into bucket
37 s3cmd get s3://BUCKET/OBJECT LOCAL_FILE
39 Get file from bucket
40 s3cmd del s3://BUCKET/OBJECT
42 Delete file from bucket
43 s3cmd rm s3://BUCKET/OBJECT
45 Delete file from bucket (alias for del)
46 s3cmd restore s3://BUCKET/OBJECT
48 Restore file from Glacier storage
49 s3cmd sync LOCAL_DIR s3://BUCKET[/PREFIX] or s3://BUCKET[/PREFIX] LO‐
51 CAL_DIR or s3://BUCKET[/PREFIX] s3://BUCKET[/PREFIX]
52 Synchronize a directory tree to S3 (checks files freshness using
53 size and md5 checksum, unless overridden by options, see below)
54 s3cmd du [s3://BUCKET[/PREFIX]]
56 Disk usage by buckets
57 s3cmd info s3://BUCKET[/OBJECT]
59 Get various information about Buckets or Files
60 s3cmd cp s3://BUCKET1/OBJECT1 s3://BUCKET2[/OBJECT2]
62 Copy object
63 s3cmd modify s3://BUCKET1/OBJECT
65 Modify object metadata
66 s3cmd mv s3://BUCKET1/OBJECT1 s3://BUCKET2[/OBJECT2]
68 Move object
69 s3cmd setacl s3://BUCKET[/OBJECT]
71 Modify Access control list for Bucket or Files
72 s3cmd setversioning s3://BUCKET enable|disable
74 Modify Bucket Versioning
75 s3cmd setobjectlegalhold STATUS s3://BUCKET/OBJECT
77 Modify Object Legal Hold
78 s3cmd setobjectretention MODE RETAIN_UNTIL_DATE s3://BUCKET/OBJECT
80 Modify Object Retention
81 s3cmd setpolicy FILE s3://BUCKET
83 Modify Bucket Policy
84 s3cmd delpolicy s3://BUCKET
86 Delete Bucket Policy
87 s3cmd setcors FILE s3://BUCKET
89 Modify Bucket CORS
90 s3cmd delcors s3://BUCKET
92 Delete Bucket CORS
93 s3cmd payer s3://BUCKET
95 Modify Bucket Requester Pays policy
96 s3cmd multipart s3://BUCKET [Id]
98 Show multipart uploads
99 s3cmd abortmp s3://BUCKET/OBJECT Id
101 Abort a multipart upload
102 s3cmd listmp s3://BUCKET/OBJECT Id
104 List parts of a multipart upload
105 s3cmd accesslog s3://BUCKET
107 Enable/disable bucket access logging
108 s3cmd sign STRING-TO-SIGN
110 Sign arbitrary string using the secret key
111 s3cmd signurl s3://BUCKET/OBJECT <expiry_epoch|+expiry_offset>
113 Sign an S3 URL to provide limited public access with expiry
114 s3cmd fixbucket s3://BUCKET[/PREFIX]
116 Fix invalid file names in a bucket
117 s3cmd expire s3://BUCKET
119 Set or delete expiration rule for the bucket
120 s3cmd setlifecycle FILE s3://BUCKET
122 Upload a lifecycle policy for the bucket
123 s3cmd getlifecycle s3://BUCKET
125 Get a lifecycle policy for the bucket
126 s3cmd dellifecycle s3://BUCKET
128 Remove a lifecycle policy for the bucket
129 s3cmd setnotification FILE s3://BUCKET
131 Upload a notification policy for the bucket
132 s3cmd getnotification s3://BUCKET
134 Get a notification policy for the bucket
135 s3cmd delnotification s3://BUCKET
137 Remove a notification policy for the bucket
138 Commands for static WebSites configuration
142 s3cmd ws-create s3://BUCKET
144 Create Website from bucket
145 s3cmd ws-delete s3://BUCKET
147 Delete Website
148 s3cmd ws-info s3://BUCKET
150 Info about Website
151 Commands for CloudFront management
155 s3cmd cflist
157 List CloudFront distribution points
158 s3cmd cfinfo [cf://DIST_ID]
160 Display CloudFront distribution point parameters
161 s3cmd cfcreate s3://BUCKET
163 Create CloudFront distribution point
164 s3cmd cfdelete cf://DIST_ID
166 Delete CloudFront distribution point
167 s3cmd cfmodify cf://DIST_ID
169 Change CloudFront distribution point parameters
170 s3cmd cfinval s3://BUCKET/OBJECT [s3://BUCKET/OBJECT ...]
172 Invalidate CloudFront objects
173 s3cmd cfinvalinfo cf://DIST_ID[/INVAL_ID]
175 Display CloudFront invalidation request(s) status
176
180 Some of the below specified options can have their default values set
181 in s3cmd config file (by default $HOME/.s3cmd). As it's a simple text
182 file feel free to open it with your favorite text editor and do any
183 changes you like.
184 -h, --help
186 show this help message and exit
187 --configure
189 Invoke interactive (re)configuration tool. Optionally use as
190 '--configure s3://some-bucket' to test access to a specific
191 bucket instead of attempting to list them all.
192 -c FILE, --config=FILE
194 Config file name. Defaults to $HOME/.s3cfg
195 --dump-config
197 Dump current configuration after parsing config files and com‐
198 mand line options and exit.
199 --access_key=ACCESS_KEY
201 AWS Access Key
202 --secret_key=SECRET_KEY
204 AWS Secret Key
205 --access_token=ACCESS_TOKEN
207 AWS Access Token
208 -n, --dry-run
210 Only show what should be uploaded or downloaded but don't actu‐
211 ally do it. May still perform S3 requests to get bucket listings
212 and other information though (only for file transfer commands)
213 -s, --ssl
215 Use HTTPS connection when communicating with S3. (default)
216 --no-ssl
218 Don't use HTTPS.
219 -e, --encrypt
221 Encrypt files before uploading to S3.
222 --no-encrypt
224 Don't encrypt files.
225 -f, --force
227 Force overwrite and other dangerous operations.
228 --continue
230 Continue getting a partially downloaded file (only for [get]
231 command).
232 --continue-put
234 Continue uploading partially uploaded files or multipart upload
235 parts. Restarts parts/files that don't have matching size and
236 md5. Skips files/parts that do. Note: md5sum checks are not
237 always sufficient to check (part) file equality. Enable this at
238 your own risk.
239 --upload-id=UPLOAD_ID
241 UploadId for Multipart Upload, in case you want continue an ex‐
242 isting upload (equivalent to --continue- put) and there are mul‐
243 tiple partial uploads. Use s3cmd multipart [URI] to see what
244 UploadIds are associated with the given URI.
245 --skip-existing
247 Skip over files that exist at the destination (only for [get]
248 and [sync] commands).
249 -r, --recursive
251 Recursive upload, download or removal.
252 --check-md5
254 Check MD5 sums when comparing files for [sync]. (default)
255 --no-check-md5
257 Do not check MD5 sums when comparing files for [sync]. Only
258 size will be compared. May significantly speed up transfer but
259 may also miss some changed files.
260 -P, --acl-public
262 Store objects with ACL allowing read for anyone.
263 --acl-private
265 Store objects with default ACL allowing access for you only.
266 --acl-grant=PERMISSION:EMAIL or USER_CANONICAL_ID
268 Grant stated permission to a given amazon user. Permission is
269 one of: read, write, read_acp, write_acp, full_control, all
270 --acl-revoke=PERMISSION:USER_CANONICAL_ID
272 Revoke stated permission for a given amazon user. Permission is
273 one of: read, write, read_acp, write_acp, full_control, all
274 -D NUM, --restore-days=NUM
276 Number of days to keep restored file available (only for 're‐
277 store' command). Default is 1 day.
278 --restore-priority=RESTORE_PRIORITY
280 Priority for restoring files from S3 Glacier (only for expedited
281 --delete-removed
283 Delete destination objects with no corresponding source file
284 [sync]
285 --no-delete-removed
287 Don't delete destination objects [sync]
288 --delete-after
290 Perform deletes AFTER new uploads when delete-removed is enabled
291 [sync]
292 --delay-updates
294 *OBSOLETE* Put all updated files into place at end [sync]
295 --max-delete=NUM
297 Do not delete more than NUM files. [del] and [sync]
298 --limit=NUM
300 Limit number of objects returned in the response body (only for
301 [ls] and [la] commands)
302 --add-destination=ADDITIONAL_DESTINATIONS
304 Additional destination for parallel uploads, in addition to last
305 arg. May be repeated.
306 --delete-after-fetch
308 Delete remote objects after fetching to local file (only for
309 [get] and [sync] commands).
310 -p, --preserve
312 Preserve filesystem attributes (mode, ownership, timestamps).
313 Default for [sync] command.
314 --no-preserve
316 Don't store FS attributes
317 --keep-dirs
319 Preserve all local directories as remote objects including empty
320 directories. Experimental feature.
321 --exclude=GLOB
323 Filenames and paths matching GLOB will be excluded from sync
324 --exclude-from=FILE
326 Read --exclude GLOBs from FILE
327 --rexclude=REGEXP
329 Filenames and paths matching REGEXP (regular expression) will be
330 excluded from sync
331 --rexclude-from=FILE
333 Read --rexclude REGEXPs from FILE
334 --include=GLOB
336 Filenames and paths matching GLOB will be included even if pre‐
337 viously excluded by one of --(r)exclude(-from) patterns
338 --include-from=FILE
340 Read --include GLOBs from FILE
341 --rinclude=REGEXP
343 Same as --include but uses REGEXP (regular expression) instead
344 of GLOB
345 --rinclude-from=FILE
347 Read --rinclude REGEXPs from FILE
348 --files-from=FILE
350 Read list of source-file names from FILE. Use - to read from
351 stdin.
352 --region=REGION, --bucket-location=REGION
354 Region to create bucket in. As of now the regions are:
355 us-east-1, us-west-1, us-west-2, eu-west-1, eu- central-1,
356 ap-northeast-1, ap-southeast-1, ap- southeast-2, sa-east-1
357 --host=HOSTNAME
359 HOSTNAME:PORT for S3 endpoint (default: s3.amazonaws.com, alter‐
360 natives such as s3-eu- west-1.amazonaws.com). You should also
361 set --host- bucket.
362 --host-bucket=HOST_BUCKET
364 DNS-style bucket+hostname:port template for accessing a bucket
365 (default: %(bucket)s.s3.amazonaws.com)
366 --reduced-redundancy, --rr
368 Store object with 'Reduced redundancy'. Lower per-GB price.
369 [put, cp, mv]
370 --no-reduced-redundancy, --no-rr
372 Store object without 'Reduced redundancy'. Higher per- GB price.
373 [put, cp, mv]
374 --storage-class=CLASS
376 Store object with specified CLASS (STANDARD, STANDARD_IA, ONE‐
377 ZONE_IA, INTELLIGENT_TIERING, GLACIER or DEEP_ARCHIVE). [put,
378 cp, mv]
379 --access-logging-target-prefix=LOG_TARGET_PREFIX
381 Target prefix for access logs (S3 URI) (for [cfmodify] and [ac‐
382 cesslog] commands)
383 --no-access-logging
385 Disable access logging (for [cfmodify] and [accesslog] commands)
386 --default-mime-type=DEFAULT_MIME_TYPE
388 Default MIME-type for stored objects. Application default is bi‐
389 nary/octet-stream.
390 -M, --guess-mime-type
392 Guess MIME-type of files by their extension or mime magic. Fall
393 back to default MIME-Type as specified by --default-mime-type
394 option
395 --no-guess-mime-type
397 Don't guess MIME-type and use the default type instead.
398 --no-mime-magic
400 Don't use mime magic when guessing MIME-type.
401 -m MIME/TYPE, --mime-type=MIME/TYPE
403 Force MIME-type. Override both --default-mime-type and
404 --guess-mime-type.
405 --add-header=NAME:VALUE
407 Add a given HTTP header to the upload request. Can be used mul‐
408 tiple times. For instance set 'Expires' or 'Cache-Control' head‐
409 ers (or both) using this option.
410 --remove-header=NAME
412 Remove a given HTTP header. Can be used multiple times. For
413 instance, remove 'Expires' or 'Cache- Control' headers (or both)
414 using this option. [modify]
415 --server-side-encryption
417 Specifies that server-side encryption will be used when putting
418 objects. [put, sync, cp, modify]
419 --server-side-encryption-kms-id=KMS_KEY
421 Specifies the key id used for server-side encryption with AWS
422 KMS-Managed Keys (SSE-KMS) when putting objects. [put, sync, cp,
423 modify]
424 --encoding=ENCODING
426 Override autodetected terminal and filesystem encoding (charac‐
427 ter set). Autodetected: UTF-8
428 --add-encoding-exts=EXTENSIONs
430 Add encoding to these comma delimited extensions i.e.
431 (css,js,html) when uploading to S3 )
432 --verbatim
434 Use the S3 name as given on the command line. No pre- process‐
435 ing, encoding, etc. Use with caution!
436 --disable-multipart
438 Disable multipart upload on files bigger than --multi‐
439 part-chunk-size-mb
440 --multipart-chunk-size-mb=SIZE
442 Size of each chunk of a multipart upload. Files bigger than SIZE
443 are automatically uploaded as multithreaded- multipart, smaller
444 files are uploaded using the traditional method. SIZE is in
445 Mega-Bytes, default chunk size is 15MB, minimum allowed chunk
446 size is 5MB, maximum is 5GB.
447 --list-md5
449 Include MD5 sums in bucket listings (only for 'ls' command).
450 --list-allow-unordered
452 Not an AWS standard. Allow the listing results to be returned in
453 unsorted order. This may be faster when listing very large buck‐
454 ets.
455 -H, --human-readable-sizes
457 Print sizes in human readable form (eg 1kB instead of 1234).
458 --ws-index=WEBSITE_INDEX
460 Name of index-document (only for [ws-create] command)
461 --ws-error=WEBSITE_ERROR
463 Name of error-document (only for [ws-create] command)
464 --expiry-date=EXPIRY_DATE
466 Indicates when the expiration rule takes effect. (only for [ex‐
467 pire] command)
468 --expiry-days=EXPIRY_DAYS
470 Indicates the number of days after object creation the expira‐
471 tion rule takes effect. (only for [expire] command)
472 --expiry-prefix=EXPIRY_PREFIX
474 Identifying one or more objects with the prefix to which the ex‐
475 piration rule applies. (only for [expire] command)
476 --skip-destination-validation
478 Skips validation of Amazon SQS, Amazon SNS, and AWS Lambda des‐
479 tinations when applying notification configuration. (only for
480 [setnotification] command)
481 --progress
483 Display progress meter (default on TTY).
484 --no-progress
486 Don't display progress meter (default on non-TTY).
487 --stats
489 Give some file-transfer stats.
490 --enable
492 Enable given CloudFront distribution (only for [cfmodify] com‐
493 mand)
494 --disable
496 Disable given CloudFront distribution (only for [cfmodify] com‐
497 mand)
498 --cf-invalidate
500 Invalidate the uploaded filed in CloudFront. Also see [cfinval]
501 command.
502 --cf-invalidate-default-index
504 When using Custom Origin and S3 static website, invalidate the
505 default index file.
506 --cf-no-invalidate-default-index-root
508 When using Custom Origin and S3 static website, don't invalidate
509 the path to the default index file.
510 --cf-add-cname=CNAME
512 Add given CNAME to a CloudFront distribution (only for [cfcre‐
513 ate] and [cfmodify] commands)
514 --cf-remove-cname=CNAME
516 Remove given CNAME from a CloudFront distribution (only for [cf‐
517 modify] command)
518 --cf-comment=COMMENT
520 Set COMMENT for a given CloudFront distribution (only for
521 [cfcreate] and [cfmodify] commands)
522 --cf-default-root-object=DEFAULT_ROOT_OBJECT
524 Set the default root object to return when no object is speci‐
525 fied in the URL. Use a relative path, i.e. default/index.html
526 instead of /default/index.html or s3://bucket/default/index.html
527 (only for [cfcreate] and [cfmodify] commands)
528 -v, --verbose
530 Enable verbose output.
531 -d, --debug
533 Enable debug output.
534 --version
536 Show s3cmd version (2.3.0-dev) and exit.
537 -F, --follow-symlinks
539 Follow symbolic links as if they are regular files
540 --cache-file=FILE
542 Cache FILE containing local source MD5 values
543 -q, --quiet
545 Silence output on stdout
546 --ca-certs=CA_CERTS_FILE
548 Path to SSL CA certificate FILE (instead of system default)
549 --ssl-cert=SSL_CLIENT_CERT_FILE
551 Path to client own SSL certificate CRT_FILE
552 --ssl-key=SSL_CLIENT_KEY_FILE
554 Path to client own SSL certificate private key KEY_FILE
555 --check-certificate
557 Check SSL certificate validity
558 --no-check-certificate
560 Do not check SSL certificate validity
561 --check-hostname
563 Check SSL certificate hostname validity
564 --no-check-hostname
566 Do not check SSL certificate hostname validity
567 --signature-v2
569 Use AWS Signature version 2 instead of newer signature methods.
570 Helpful for S3-like systems that don't have AWS Signature v4
571 yet.
572 --limit-rate=LIMITRATE
574 Limit the upload or download speed to amount bytes per second.
575 Amount may be expressed in bytes, kilobytes with the k suffix,
576 or megabytes with the m suffix
577 --no-connection-pooling
579 Disable connection reuse
580 --requester-pays
582 Set the REQUESTER PAYS flag for operations
583 -l, --long-listing
585 Produce long listing [ls]
586 --stop-on-error
588 stop if error in transfer
589 --max-retries=NUM
591 Maximum number of times to retry a failed request before giving
592 up. Default is 5
593 --content-disposition=CONTENT_DISPOSITION
595 Provide a Content-Disposition for signed URLs, e.g., "inline;
596 filename=myvideo.mp4"
597 --content-type=CONTENT_TYPE
599 Provide a Content-Type for signed URLs, e.g., "video/mp4"
600
604 One of the most powerful commands of s3cmd is s3cmd sync used for syn‐
605 chronising complete directory trees to or from remote S3 storage. To
606 some extent s3cmd put and s3cmd get share a similar behaviour with
607 sync.
608 Basic usage common in backup scenarios is as simple as:
610 s3cmd sync /local/path/ s3://test-bucket/backup/
611 This command will find all files under /local/path directory and copy
613 them to corresponding paths under s3://test-bucket/backup on the remote
614 side. For example:
615 /local/path/file1.ext -> s3://bucket/backup/file1.ext
616 /local/path/dir123/file2.bin -> s3://bucket/backup/dir123/file2.bin
617 However if the local path doesn't end with a slash the last directory's
619 name is used on the remote side as well. Compare these with the previ‐
620 ous example:
621 s3cmd sync /local/path s3://test-bucket/backup/
622 will sync:
623 /local/path/file1.ext -> s3://bucket/backup/path/file1.ext
624 /local/path/dir123/file2.bin -> s3://bucket/backup/path/dir123/file2.bin
625 To retrieve the files back from S3 use inverted syntax:
627 s3cmd sync s3://test-bucket/backup/ ~/restore/
628 that will download files:
629 s3://bucket/backup/file1.ext -> ~/restore/file1.ext
630 s3://bucket/backup/dir123/file2.bin -> ~/restore/dir123/file2.bin
631 Without the trailing slash on source the behaviour is similar to what
633 has been demonstrated with upload:
634 s3cmd sync s3://test-bucket/backup ~/restore/
635 will download the files as:
636 s3://bucket/backup/file1.ext -> ~/restore/backup/file1.ext
637 s3://bucket/backup/dir123/file2.bin -> ~/restore/backup/dir123/file2.bin
638 All source file names, the bold ones above, are matched against exclude
640 rules and those that match are then re-checked against include rules to
641 see whether they should be excluded or kept in the source list.
642 For the purpose of --exclude and --include matching only the bold file
644 names above are used. For instance only path/file1.ext is tested
645 against the patterns, not /local/path/file1.ext
646 Both --exclude and --include work with shell-style wildcards (a.k.a.
648 GLOB). For a greater flexibility s3cmd provides Regular-expression
649 versions of the two exclude options named --rexclude and --rinclude.
650 The options with ...-from suffix (eg --rinclude-from) expect a filename
651 as an argument. Each line of such a file is treated as one pattern.
652 There is only one set of patterns built from all --(r)exclude(-from)
654 options and similarly for include variant. Any file excluded with eg
655 --exclude can be put back with a pattern found in --rinclude-from list.
656 Run s3cmd with --dry-run to verify that your rules work as expected.
658 Use together with --debug get detailed information about matching file
659 names against exclude and include rules.
660 For example to exclude all files with ".jpg" extension except those be‐
662 ginning with a number use:
663 --exclude '*.jpg' --rinclude '[0-9].*.jpg'
665 To exclude all files except "*.jpg" extension, use:
667 --exclude '*' --include '*.jpg'
669 To exclude local directory 'somedir', be sure to use a trailing forward
671 slash, as such:
672 --exclude 'somedir/'
674
676 For the most up to date list of options run: s3cmd --help
677 For more info about usage, examples and other related info visit
678 project homepage at: https://s3tools.org
679
681 Written by Michal Ludvig, Florent Viard and contributors
682
684 Preferred way to get support is our mailing list:
685 s3tools-general@lists.sourceforge.net
686 or visit the project homepage:
687 https://s3tools.org
688
690 Report bugs to s3tools-bugs@lists.sourceforge.net
691
693 Copyright © 2007-2023 TGRMN Software (https://www.tgrmn.com), Sodria
694 SAS (https://www.sodria.com) and contributors
695
697 This program is free software; you can redistribute it and/or modify it
698 under the terms of the GNU General Public License as published by the
699 Free Software Foundation; either version 2 of the License, or (at your
700 option) any later version. This program is distributed in the hope
701 that it will be useful, but WITHOUT ANY WARRANTY; without even the im‐
702 plied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
703 See the GNU General Public License for more details.
704 s3cmd(1)