1containers-storage.conf(5)(Container)Filecontainers-storage.conf(5)(Container)
2Dan Walsh May 2017
6
9 storage.conf - Syntax of Container Storage configuration file
10
13 The STORAGE configuration file specifies all of the available container
14 storage options for tools using shared container storage, but in a TOML
15 format that can be more easily modified and versioned.
16
19 The [TOML format][toml] is used as the encoding of the configuration
20 file. Every option and subtable listed here is nested under a global
21 "storage" table. No bare options are used. The format of TOML can be
22 simplified to:
23 [table]
26 option = value
27 [table.subtable1]
29 option = value
30 [table.subtable2]
32 option = value
33
37 The storage table supports the following options:
38 driver=""
41 Copy On Write (COW) container storage driver. Valid drivers are
42 "overlay", "vfs", "devmapper", "aufs", "btrfs", and "zfs". Some drivers
43 (for example, "zfs", "btrfs", and "aufs") may not work if your kernel
44 lacks support for the filesystem. This field is required to guarantee
45 proper operation. Valid rootless drivers are "btrfs", "overlay", and
46 "vfs". Rootless users default to the driver defined in the system con‐
47 figuration when possible. When the system configuration uses an unsup‐
48 ported rootless driver, rootless users default to "overlay" if avail‐
49 able, otherwise "vfs".
50 graphroot=""
53 container storage graph dir (default: "/var/lib/containers/storage")
54 Default directory to store all writable content created by container
55 storage programs. The rootless graphroot path supports environment
56 variable substitutions (ie. $HOME/containers/storage). When changing
57 the graphroot location on an SELINUX system, ensure the labeling
58 matches the default locations labels with the following commands:
59 # semanage fcontext -a -e /var/lib/containers/storage /NEWSTORAGEPATH
62 # restorecon -R -v /NEWSTORAGEPATH
63 In rootless mode you would set
67 # semanage fcontext -a -e $HOME/.local/share/containers NEWSTORAGEPATH
70 $ restorecon -R -v /NEWSTORAGEPATH
71 rootless_storage_path="$HOME/.local/share/containers/storage"
75 Storage path for rootless users. By default the graphroot for root‐
76 less users is set to $XDG_DATA_HOME/containers/storage, if
77 XDG_DATA_HOME is set. Otherwise $HOME/.local/share/containers/storage
78 is used. This field can be used if administrators need to change the
79 storage location for all users. The rootless storage path supports en‐
80 vironment variable substitutions (ie. $HOME/containers/storage)
81 A common use case for this field is to provide a local storage direc‐
84 tory when user home directories are NFS-mounted (podman does not sup‐
85 port container storage over NFS).
86 runroot=""
89 container storage run dir (default: "/run/containers/storage") De‐
90 fault directory to store all temporary writable content created by con‐
91 tainer storage programs. The rootless runroot path supports environment
92 variable substitutions (ie. $HOME/containers/storage)
93 driver_priority=[]
96 Priority list for the storage drivers that will be tested one after
97 the other to pick the storage driver if it is not defined. The first
98 storage driver in this list that can be used, will be picked as the new
99 one and all subsequent ones will not be tried. If all drivers in this
100 list are not viable, then all known drivers will be tried and the first
101 working one will be picked. By default, the storage driver is set via
102 the driver option. If it is not defined, then the best driver will be
103 picked according to the current platform. This option allows you to
104 override this internal priority list with a custom one to prefer cer‐
105 tain drivers. Setting this option only has an effect if the local
106 storage has not been initialized yet and the driver name is not set.
107 STORAGE OPTIONS TABLE
110 The storage.options table supports the following options:
111 additionalimagestores=[]
114 Paths to additional container image stores. Usually these are
115 read/only and stored on remote network shares.
116 pull_options = {enable_partial_images = "false", use_hard_links =
119 "false", ostree_repos=""}
120 Allows specification of how storage is populated when pulling images.
123 This option can speed the pulling process of images compressed with
124 format zstd:chunked. Containers/storage looks for files within images
125 that are being pulled from a container registry that were previously
126 pulled to the host. It can copy or create a hard link to the existing
127 file when it finds them, eliminating the need to pull them from the
128 container registry. These options can deduplicate pulling of content,
129 disk storage of content and can allow the kernel to use less memory
130 when running containers.
131 containers/storage supports four keys
134 * enable_partial_images="true" | "false"
135 Tells containers/storage to look for files previously pulled in
136 storage
137 rather then always pulling them from the container registry.
138 * use_hard_links = "false" | "true"
139 Tells containers/storage to use hard links rather then create new
140 files in
141 the image, if an identical file already existed in storage.
142 * ostree_repos = ""
143 Tells containers/storage where an ostree repository exists that
144 might have
145 previously pulled content which can be used when attempting to
146 avoid
147 pulling content from the container registry
148 remap-uids="" remap-gids=""
151 Remap-UIDs/GIDs is the mapping from UIDs/GIDs as they should appear
152 inside of a container, to the UIDs/GIDs outside of the container, and
153 the length of the range of UIDs/GIDs. Additional mapped sets can be
154 listed and will be heeded by libraries, but there are limits to the
155 number of mappings which the kernel will allow when you later attempt
156 to run a container.
157 Example
160 remap-uids = 0:1668442479:65536
161 remap-gids = 0:1668442479:65536
162 These mappings tell the container engines to map UID 0 inside of the
165 container to UID 1668442479 outside. UID 1 will be mapped to
166 1668442480. UID 2 will be mapped to 1668442481, etc, for the next 65533
167 UIDs in succession.
168 remap-user="" remap-group=""
171 Remap-User/Group is a user name which can be used to look up one or
172 more UID/GID ranges in the /etc/subuid or /etc/subgid file. Mappings
173 are set up starting with an in-container ID of 0 and then a host-level
174 ID taken from the lowest range that matches the specified name, and us‐
175 ing the length of that range. Additional ranges are then assigned, us‐
176 ing the ranges which specify the lowest host-level IDs first, to the
177 lowest not-yet-mapped in-container ID, until all of the entries have
178 been used for maps.
179 Example
182 remap-user = "containers"
183 remap-group = "containers"
184 root-auto-userns-user=""
187 Root-auto-userns-user is a user name which can be used to look up one
188 or more UID/GID ranges in the /etc/subuid and /etc/subgid file. These
189 ranges will be partitioned to containers configured to create automati‐
190 cally a user namespace. Containers configured to automatically create
191 a user namespace can still overlap with containers having an explicit
192 mapping set. This setting is ignored when running as rootless.
193 auto-userns-min-size=1024
196 Auto-userns-min-size is the minimum size for a user namespace created
197 automatically.
198 auto-userns-max-size=65536
201 Auto-userns-max-size is the maximum size for a user namespace created
202 automatically.
203 disable-volatile=true
206 If disable-volatile is set, then the "volatile" mount optimization is
207 disabled for all the containers.
208 STORAGE OPTIONS FOR AUFS TABLE
211 The storage.options.aufs table supports the following options:
212 mountopt=""
215 Comma separated list of default options to be used to mount container
216 images. Suggested value "nodev". Mount options are documented in the
217 mount(8) man page.
218 STORAGE OPTIONS FOR BTRFS TABLE
221 The storage.options.btrfs table supports the following options:
222 min_space=""
225 Specifies the min space in a btrfs volume.
226 size=""
229 Maximum size of a container image. This flag can be used to set
230 quota on the size of container images. (format: [], where unit = b
231 (bytes), k (kilobytes), m (megabytes), or g (gigabytes))
232 STORAGE OPTIONS FOR THINPOOL (devicemapper) TABLE
235 The storage.options.thinpool table supports the following options for
236 the devicemapper driver:
237 autoextend_percent=""
240 Tells the thinpool driver the amount by which the thinpool needs to
241 be grown. This is specified in terms of % of pool size. So a value of
242 20 means that when threshold is hit, pool will be grown by 20% of ex‐
243 isting pool size. (default: 20%)
244 autoextend_threshold=""
247 Tells the driver the thinpool extension threshold in terms of per‐
248 centage of pool size. For example, if threshold is 60, that means when
249 pool is 60% full, threshold has been hit. (default: 80%)
250 basesize=""
253 Specifies the size to use when creating the base device, which limits
254 the size of images and containers. (default: 10g)
255 blocksize=""
258 Specifies a custom blocksize to use for the thin pool. (default: 64k)
259 directlvm_device=""
262 Specifies a custom block storage device to use for the thin pool. Re‐
263 quired for using graphdriver devicemapper.
264 directlvm_device_force=""
267 Tells driver to wipe device (directlvm_device) even if device already
268 has a filesystem. (default: false)
269 fs="xfs"
272 Specifies the filesystem type to use for the base device. (default:
273 xfs)
274 log_level=""
277 Sets the log level of devicemapper.
278 0: LogLevelSuppress 0 (default)
281 2: LogLevelFatal
282 3: LogLevelErr
283 4: LogLevelWarn
284 5: LogLevelNotice
285 6: LogLevelInfo
286 7: LogLevelDebug
287 metadata_size=""
291 metadata_size is used to set the pvcreate --metadatasize options when
292 creating thin devices. (Default 128k)
293 min_free_space=""
296 Specifies the min free space percent in a thin pool required for new
297 device creation to succeed. Valid values are from 0% - 99%. Value 0%
298 disables. (default: 10%)
299 mkfsarg=""
302 Specifies extra mkfs arguments to be used when creating the base de‐
303 vice.
304 mountopt=""
307 Comma separated list of default options to be used to mount container
308 images. Suggested value "nodev". Mount options are documented in the
309 mount(8) man page.
310 size=""
313 Maximum size of a container image. This flag can be used to set
314 quota on the size of container images. (format: [], where unit = b
315 (bytes), k (kilobytes), m (megabytes), or g (gigabytes))
316 use_deferred_deletion=""
319 Marks thinpool device for deferred deletion. If the thinpool is in
320 use when the driver attempts to delete it, the driver will attempt to
321 delete device every 30 seconds until successful, or when it restarts.
322 Deferred deletion permanently deletes the device and all data stored in
323 the device will be lost. (default: true).
324 use_deferred_removal=""
327 Marks devicemapper block device for deferred removal. If the device
328 is in use when its driver attempts to remove it, the driver tells the
329 kernel to remove the device as soon as possible. Note this does not
330 free up the disk space, use deferred deletion to fully remove the thin‐
331 pool. (default: true).
332 xfs_nospace_max_retries=""
335 Specifies the maximum number of retries XFS should attempt to com‐
336 plete IO when ENOSPC (no space) error is returned by underlying storage
337 device. (default: 0, which means to try continuously.)
338 STORAGE OPTIONS FOR OVERLAY TABLE
341 The storage.options.overlay table supports the following options:
342 ignore_chown_errors = "false"
345 ignore_chown_errors can be set to allow a non privileged user running
346 with a single UID within a user namespace to run containers. The user
347 can pull and use any image even those with multiple uids. Note multi‐
348 ple UIDs will be squashed down to the default uid in the container.
349 These images will have no separation between the users in the con‐
350 tainer. (default: false)
351 inodes=""
354 Maximum inodes in a read/write layer. This flag can be used to set
355 a quota on the inodes allocated for a read/write layer of a container.
356 force_mask = "0000|shared|private"
359 ForceMask specifies the permissions mask that is used for new files
360 and directories. The values "shared" and "private" are accepted. (de‐
361 fault: ""). Octal permission masks are also accepted.
362 • ``: Not set All files/directories, get set with the permis‐
365 sions identified within the image.
366 • private: it is equivalent to 0700. All files/directories get
368 set with 0700 permissions. The owner has rwx access to the
369 files. No other users on the system can access the files.
370 This setting could be used with networked based home directo‐
371 ries.
372 • shared: it is equivalent to 0755. The owner has rwx access to
374 the files and everyone else can read, access and execute them.
375 This setting is useful for sharing containers storage with
376 other users. For instance, a storage owned by root could be
377 shared to rootless users as an additional store. NOTE: All
378 files within the image are made readable and executable by any
379 user on the system. Even /etc/shadow within your image is now
380 readable by any user.
381 OCTAL: Users can experiment with other OCTAL Permissions.
385 Note: The force_mask Flag is an experimental feature, it could change
388 in the future. When "force_mask" is set the original permission mask
389 is stored in the "user.containers.override_stat" xattr and the
390 "mount_program" option must be specified. Mount programs like
391 "/usr/bin/fuse-overlayfs" present the extended attribute permissions to
392 processes within containers rather than the "force_mask" permissions.
393 mount_program=""
396 Specifies the path to a custom program to use instead of using kernel
397 defaults for mounting the file system. In rootless mode, without the
398 CAP_SYS_ADMIN capability, many kernels prevent mounting of overlay file
399 systems, requiring you to specify a mount_program. The mount_program
400 option is also required on systems where the underlying storage is
401 btrfs, aufs, zfs, overlay, or ecryptfs based file systems.
402 mount_program = "/usr/bin/fuse-overlayfs"
403 mountopt=""
406 Comma separated list of default options to be used to mount container
407 images. Suggested value "nodev". Mount options are documented in the
408 mount(8) man page.
409 skip_mount_home=""
412 Tell storage drivers to not create a PRIVATE bind mount on their home
413 directory.
414 size=""
417 Maximum size of a read/write layer. This flag can be used to set
418 quota on the size of a read/write layer of a container. (format: [],
419 where unit = b (bytes), k (kilobytes), m (megabytes), or g (gigabytes))
420 STORAGE OPTIONS FOR VFS TABLE
423 The storage.options.vfs table supports the following options:
424 ignore_chown_errors = "false"
427 ignore_chown_errors can be set to allow a non privileged user running
428 with a single UID within a user namespace to run containers. The user
429 can pull and use any image even those with multiple uids. Note multi‐
430 ple UIDs will be squashed down to the default uid in the container.
431 These images will have no separation between the users in the con‐
432 tainer. (default: false)
433 STORAGE OPTIONS FOR ZFS TABLE
436 The storage.options.zfs table supports the following options:
437 fsname=""
440 File System name for the zfs driver
441 mountopt=""
444 Comma separated list of default options to be used to mount container
445 images. Suggested value "nodev". Mount options are documented in the
446 mount(8) man page.
447 size=""
450 Maximum size of a container image. This flag can be used to set
451 quota on the size of container images. (format: [], where unit = b
452 (bytes), k (kilobytes), m (megabytes), or g (gigabytes))
453
456 When running on an SELinux system, if you move the containers storage
457 graphroot directory, you must make sure the labeling is correct.
458 Tell SELinux about the new containers storage by setting up an equiva‐
461 lence record. This tells SELinux to label content under the new path,
462 as if it was stored under /var/lib/containers/storage.
463 semanage fcontext -a -e /var/lib/containers NEWSTORAGEPATH
466 restorecon -R -v NEWSTORAGEPATH
467 In rootless mode, you would set
471 semanage fcontext -a -e $HOME/.local/share/containers NEWSTORAGEPATH
474 restorecon -R -v NEWSTORAGEPATH
475 The semanage command above tells SELinux to setup the default labeling
479 of NEWSTORAGEPATH to match /var/lib/containers. The restorecon command
480 tells SELinux to apply the labels to the actual content.
481 Now all new content created in these directories will automatically be
484 created with the correct label.
485
488 Container storage implements XFS project quota controls for overlay
489 storage containers and volumes. The directory used to store the con‐
490 tainers must be an XFS file system and be mounted with the pquota op‐
491 tion.
492 Example /etc/fstab entry:
495 /dev/podman/podman-var /var xfs defaults,x-systemd.device-timeout=0,pquota 1 2
498 Container storage generates project ids for each container and builtin
502 volume, but these project ids need to be unique for the XFS file sys‐
503 tem.
504 The xfs_quota tool can be used to assign a project id to the storage
507 driver directory, e.g.:
508 echo 100000:/var/lib/containers/storage/overlay >> /etc/projects
511 echo 200000:/var/lib/containers/storage/volumes >> /etc/projects
512 echo storage:100000 >> /etc/projid
513 echo volumes:200000 >> /etc/projid
514 xfs_quota -x -c 'project -s storage volumes' /<xfs mount point>
515 In the example above, the storage directory project id will be used as
519 a "start offset" and all containers will be assigned larger project ids
520 (e.g. >= 100000). Then the volumes directory project id will be used
521 as a "start offset" and all volumes will be assigned larger project ids
522 (e.g. >= 200000). This is a way to prevent xfs_quota management from
523 conflicting with containers/storage.
524
527 Distributions often provide a /usr/share/containers/storage.conf file
528 to define default storage configuration. Administrators can override
529 this file by creating /etc/containers/storage.conf to specify their own
530 configuration. Likewise rootless users can create a storage.conf file
531 to override the system storage.conf files. Files should be stored in
532 the $XDG_CONFIG_HOME/containers/storage.conf file. If $XDG_CONFIG_HOME
533 is not set then the file $HOME/.config/containers/storage.conf is used.
534 Note: The storage.conf file overrides all other storage.conf files.
537 Container engines run by users with a storage.conf file in their home
538 directory do not use options in the system storage.conf files.
539 /etc/projects - XFS persistent project root definition /etc/projid -
542 XFS project name mapping file
543
546 semanage(8), restorecon(8), mount(8), fuse-overlayfs(1), xfs_quota(8),
547 projects(5), projid(5)
548
551 May 2017, Originally compiled by Dan Walsh dwalsh@redhat.com
552 ⟨mailto:dwalsh@redhat.com⟩ Format copied from crio.conf man page cre‐
553 ated by Aleksa Sarai asarai@suse.de ⟨mailto:asarai@suse.de⟩
554Configuration Storagceontainers-storage.conf(5)(Container)