1containers-storage.conf(5)(Container)Filecontainers-storage.conf(5)(Container)
2Dan Walsh May 2017
6
9 storage.conf - Syntax of Container Storage configuration file
10
13 The STORAGE configuration file specifies all of the available container
14 storage options for tools using shared container storage, but in a TOML
15 format that can be more easily modified and versioned.
16
19 The [TOML format][toml] is used as the encoding of the configuration
20 file. Every option and subtable listed here is nested under a global
21 "storage" table. No bare options are used. The format of TOML can be
22 simplified to:
23 [table]
26 option = value
27 [table.subtable1]
29 option = value
30 [table.subtable2]
32 option = value
33
37 The storage table supports the following options:
38 driver=""
41 Copy On Write (COW) container storage driver. Valid drivers are
42 "overlay", "vfs", "devmapper", "aufs", "btrfs", and "zfs". Some drivers
43 (for example, "zfs", "btrfs", and "aufs") may not work if your kernel
44 lacks support for the filesystem. This field is required to guarantee
45 proper operation. Valid rootless drivers are "btrfs", "overlay", and
46 "vfs". Rootless users default to the driver defined in the system con‐
47 figuration when possible. When the system configuration uses an unsup‐
48 ported rootless driver, rootless users default to "overlay" if avail‐
49 able, otherwise "vfs".
50 graphroot=""
53 container storage graph dir (default: "/var/lib/containers/storage")
54 Default directory to store all writable content created by container
55 storage programs. The rootless graphroot path supports environment
56 variable substitutions (ie. $HOME/containers/storage). When changing
57 the graphroot location on an SELINUX system, ensure the labeling
58 matches the default locations labels with the following commands:
59 # semanage fcontext -a -e /var/lib/containers/storage /NEWSTORAGEPATH
62 # restorecon -R -v /NEWSTORAGEPATH
63 In rootless mode you would set
67 # semanage fcontext -a -e $HOME/.local/share/containers NEWSTORAGEPATH
70 $ restorecon -R -v /NEWSTORAGEPATH
71 rootless_storage_path="$HOME/.local/share/containers/storage"
75 Storage path for rootless users. By default the graphroot for root‐
76 less users is set to $XDG_DATA_HOME/containers/storage, if
77 XDG_DATA_HOME is set. Otherwise $HOME/.local/share/containers/storage
78 is used. This field can be used if administrators need to change the
79 storage location for all users. The rootless storage path supports en‐
80 vironment variable substitutions (ie. $HOME/containers/storage)
81 A common use case for this field is to provide a local storage direc‐
84 tory when user home directories are NFS-mounted (podman does not sup‐
85 port container storage over NFS).
86 runroot=""
89 container storage run dir (default: "/run/containers/storage") De‐
90 fault directory to store all temporary writable content created by con‐
91 tainer storage programs. The rootless runroot path supports environment
92 variable substitutions (ie. $HOME/containers/storage)
93 driver_priority=[]
96 Priority list for the storage drivers that will be tested one after
97 the other to pick the storage driver if it is not defined. The first
98 storage driver in this list that can be used, will be picked as the new
99 one and all subsequent ones will not be tried. If all drivers in this
100 list are not viable, then all known drivers will be tried and the first
101 working one will be picked. By default, the storage driver is set via
102 the driver option. If it is not defined, then the best driver will be
103 picked according to the current platform. This option allows you to
104 override this internal priority list with a custom one to prefer cer‐
105 tain drivers. Setting this option only has an effect if the local
106 storage has not been initialized yet and the driver name is not set.
107 transient_store = "false" | "true"
110 Transient store mode makes all container metadata be saved in temporary
113 storage (i.e. runroot above). This is faster, but doesn't persist
114 across reboots. Additional garbage collection must also be performed
115 at boot-time, so this option should remain disabled in most configura‐
116 tions. (default: false)
117 STORAGE OPTIONS TABLE
120 The storage.options table supports the following options:
121 additionalimagestores=[]
124 Paths to additional container image stores. Usually these are
125 read/only and stored on remote network shares.
126 pull_options = {enable_partial_images = "false", use_hard_links =
129 "false", ostree_repos=""}
130 Allows specification of how storage is populated when pulling images.
133 This option can speed the pulling process of images compressed with
134 format zstd:chunked. Containers/storage looks for files within images
135 that are being pulled from a container registry that were previously
136 pulled to the host. It can copy or create a hard link to the existing
137 file when it finds them, eliminating the need to pull them from the
138 container registry. These options can deduplicate pulling of content,
139 disk storage of content and can allow the kernel to use less memory
140 when running containers.
141 containers/storage supports four keys
144 * enable_partial_images="true" | "false"
145 Tells containers/storage to look for files previously pulled in
146 storage
147 rather then always pulling them from the container registry.
148 * use_hard_links = "false" | "true"
149 Tells containers/storage to use hard links rather then create new
150 files in
151 the image, if an identical file already existed in storage.
152 * ostree_repos = ""
153 Tells containers/storage where an ostree repository exists that
154 might have
155 previously pulled content which can be used when attempting to
156 avoid
157 pulling content from the container registry
158 remap-uids="" remap-gids=""
161 Remap-UIDs/GIDs is the mapping from UIDs/GIDs as they should appear
162 inside of a container, to the UIDs/GIDs outside of the container, and
163 the length of the range of UIDs/GIDs. Additional mapped sets can be
164 listed and will be heeded by libraries, but there are limits to the
165 number of mappings which the kernel will allow when you later attempt
166 to run a container.
167 Example
170 remap-uids = 0:1668442479:65536
171 remap-gids = 0:1668442479:65536
172 These mappings tell the container engines to map UID 0 inside of the
175 container to UID 1668442479 outside. UID 1 will be mapped to
176 1668442480. UID 2 will be mapped to 1668442481, etc, for the next 65533
177 UIDs in succession.
178 remap-user="" remap-group=""
181 Remap-User/Group is a user name which can be used to look up one or
182 more UID/GID ranges in the /etc/subuid or /etc/subgid file. Mappings
183 are set up starting with an in-container ID of 0 and then a host-level
184 ID taken from the lowest range that matches the specified name, and us‐
185 ing the length of that range. Additional ranges are then assigned, us‐
186 ing the ranges which specify the lowest host-level IDs first, to the
187 lowest not-yet-mapped in-container ID, until all of the entries have
188 been used for maps.
189 Example
192 remap-user = "containers"
193 remap-group = "containers"
194 root-auto-userns-user=""
197 Root-auto-userns-user is a user name which can be used to look up one
198 or more UID/GID ranges in the /etc/subuid and /etc/subgid file. These
199 ranges will be partitioned to containers configured to create automati‐
200 cally a user namespace. Containers configured to automatically create
201 a user namespace can still overlap with containers having an explicit
202 mapping set. This setting is ignored when running as rootless.
203 auto-userns-min-size=1024
206 Auto-userns-min-size is the minimum size for a user namespace created
207 automatically.
208 auto-userns-max-size=65536
211 Auto-userns-max-size is the maximum size for a user namespace created
212 automatically.
213 disable-volatile=true
216 If disable-volatile is set, then the "volatile" mount optimization is
217 disabled for all the containers.
218 STORAGE OPTIONS FOR AUFS TABLE
221 The storage.options.aufs table supports the following options:
222 mountopt=""
225 Comma separated list of default options to be used to mount container
226 images. Suggested value "nodev". Mount options are documented in the
227 mount(8) man page.
228 STORAGE OPTIONS FOR BTRFS TABLE
231 The storage.options.btrfs table supports the following options:
232 min_space=""
235 Specifies the min space in a btrfs volume.
236 size=""
239 Maximum size of a container image. This flag can be used to set
240 quota on the size of container images. (format: [], where unit = b
241 (bytes), k (kilobytes), m (megabytes), or g (gigabytes))
242 STORAGE OPTIONS FOR THINPOOL (devicemapper) TABLE
245 The storage.options.thinpool table supports the following options for
246 the devicemapper driver:
247 autoextend_percent=""
250 Tells the thinpool driver the amount by which the thinpool needs to
251 be grown. This is specified in terms of % of pool size. So a value of
252 20 means that when threshold is hit, pool will be grown by 20% of ex‐
253 isting pool size. (default: 20%)
254 autoextend_threshold=""
257 Tells the driver the thinpool extension threshold in terms of per‐
258 centage of pool size. For example, if threshold is 60, that means when
259 pool is 60% full, threshold has been hit. (default: 80%)
260 basesize=""
263 Specifies the size to use when creating the base device, which limits
264 the size of images and containers. (default: 10g)
265 blocksize=""
268 Specifies a custom blocksize to use for the thin pool. (default: 64k)
269 directlvm_device=""
272 Specifies a custom block storage device to use for the thin pool. Re‐
273 quired for using graphdriver devicemapper.
274 directlvm_device_force=""
277 Tells driver to wipe device (directlvm_device) even if device already
278 has a filesystem. (default: false)
279 fs="xfs"
282 Specifies the filesystem type to use for the base device. (default:
283 xfs)
284 log_level=""
287 Sets the log level of devicemapper.
288 0: LogLevelSuppress 0 (default)
291 2: LogLevelFatal
292 3: LogLevelErr
293 4: LogLevelWarn
294 5: LogLevelNotice
295 6: LogLevelInfo
296 7: LogLevelDebug
297 metadata_size=""
301 metadata_size is used to set the pvcreate --metadatasize options when
302 creating thin devices. (Default 128k)
303 min_free_space=""
306 Specifies the min free space percent in a thin pool required for new
307 device creation to succeed. Valid values are from 0% - 99%. Value 0%
308 disables. (default: 10%)
309 mkfsarg=""
312 Specifies extra mkfs arguments to be used when creating the base de‐
313 vice.
314 mountopt=""
317 Comma separated list of default options to be used to mount container
318 images. Suggested value "nodev". Mount options are documented in the
319 mount(8) man page.
320 size=""
323 Maximum size of a container image. This flag can be used to set
324 quota on the size of container images. (format: [], where unit = b
325 (bytes), k (kilobytes), m (megabytes), or g (gigabytes))
326 use_deferred_deletion=""
329 Marks thinpool device for deferred deletion. If the thinpool is in
330 use when the driver attempts to delete it, the driver will attempt to
331 delete device every 30 seconds until successful, or when it restarts.
332 Deferred deletion permanently deletes the device and all data stored in
333 the device will be lost. (default: true).
334 use_deferred_removal=""
337 Marks devicemapper block device for deferred removal. If the device
338 is in use when its driver attempts to remove it, the driver tells the
339 kernel to remove the device as soon as possible. Note this does not
340 free up the disk space, use deferred deletion to fully remove the thin‐
341 pool. (default: true).
342 xfs_nospace_max_retries=""
345 Specifies the maximum number of retries XFS should attempt to com‐
346 plete IO when ENOSPC (no space) error is returned by underlying storage
347 device. (default: 0, which means to try continuously.)
348 STORAGE OPTIONS FOR OVERLAY TABLE
351 The storage.options.overlay table supports the following options:
352 ignore_chown_errors = "false"
355 ignore_chown_errors can be set to allow a non privileged user running
356 with a single UID within a user namespace to run containers. The user
357 can pull and use any image even those with multiple uids. Note multi‐
358 ple UIDs will be squashed down to the default uid in the container.
359 These images will have no separation between the users in the con‐
360 tainer. (default: false)
361 inodes=""
364 Maximum inodes in a read/write layer. This flag can be used to set
365 a quota on the inodes allocated for a read/write layer of a container.
366 force_mask = "0000|shared|private"
369 ForceMask specifies the permissions mask that is used for new files
370 and directories. The values "shared" and "private" are accepted. (de‐
371 fault: ""). Octal permission masks are also accepted.
372 • ``: Not set All files/directories, get set with the permis‐
375 sions identified within the image.
376 • private: it is equivalent to 0700. All files/directories get
378 set with 0700 permissions. The owner has rwx access to the
379 files. No other users on the system can access the files.
380 This setting could be used with networked based home directo‐
381 ries.
382 • shared: it is equivalent to 0755. The owner has rwx access to
384 the files and everyone else can read, access and execute them.
385 This setting is useful for sharing containers storage with
386 other users. For instance, a storage owned by root could be
387 shared to rootless users as an additional store. NOTE: All
388 files within the image are made readable and executable by any
389 user on the system. Even /etc/shadow within your image is now
390 readable by any user.
391 OCTAL: Users can experiment with other OCTAL Permissions.
395 Note: The force_mask Flag is an experimental feature, it could change
398 in the future. When "force_mask" is set the original permission mask
399 is stored in the "user.containers.override_stat" xattr and the
400 "mount_program" option must be specified. Mount programs like
401 "/usr/bin/fuse-overlayfs" present the extended attribute permissions to
402 processes within containers rather than the "force_mask" permissions.
403 mount_program=""
406 Specifies the path to a custom program to use instead of using kernel
407 defaults for mounting the file system. In rootless mode, without the
408 CAP_SYS_ADMIN capability, many kernels prevent mounting of overlay file
409 systems, requiring you to specify a mount_program. The mount_program
410 option is also required on systems where the underlying storage is
411 btrfs, aufs, zfs, overlay, or ecryptfs based file systems.
412 mount_program = "/usr/bin/fuse-overlayfs"
413 mountopt=""
416 Comma separated list of default options to be used to mount container
417 images. Suggested value "nodev". Mount options are documented in the
418 mount(8) man page.
419 skip_mount_home=""
422 Tell storage drivers to not create a PRIVATE bind mount on their home
423 directory.
424 size=""
427 Maximum size of a read/write layer. This flag can be used to set
428 quota on the size of a read/write layer of a container. (format: [],
429 where unit = b (bytes), k (kilobytes), m (megabytes), or g (gigabytes))
430 STORAGE OPTIONS FOR VFS TABLE
433 The storage.options.vfs table supports the following options:
434 ignore_chown_errors = "false"
437 ignore_chown_errors can be set to allow a non privileged user running
438 with a single UID within a user namespace to run containers. The user
439 can pull and use any image even those with multiple uids. Note multi‐
440 ple UIDs will be squashed down to the default uid in the container.
441 These images will have no separation between the users in the con‐
442 tainer. (default: false)
443 STORAGE OPTIONS FOR ZFS TABLE
446 The storage.options.zfs table supports the following options:
447 fsname=""
450 File System name for the zfs driver
451 mountopt=""
454 Comma separated list of default options to be used to mount container
455 images. Suggested value "nodev". Mount options are documented in the
456 mount(8) man page.
457 size=""
460 Maximum size of a container image. This flag can be used to set
461 quota on the size of container images. (format: [], where unit = b
462 (bytes), k (kilobytes), m (megabytes), or g (gigabytes))
463
466 When running on an SELinux system, if you move the containers storage
467 graphroot directory, you must make sure the labeling is correct.
468 Tell SELinux about the new containers storage by setting up an equiva‐
471 lence record. This tells SELinux to label content under the new path,
472 as if it was stored under /var/lib/containers/storage.
473 semanage fcontext -a -e /var/lib/containers NEWSTORAGEPATH
476 restorecon -R -v NEWSTORAGEPATH
477 In rootless mode, you would set
481 semanage fcontext -a -e $HOME/.local/share/containers NEWSTORAGEPATH
484 restorecon -R -v NEWSTORAGEPATH
485 The semanage command above tells SELinux to setup the default labeling
489 of NEWSTORAGEPATH to match /var/lib/containers. The restorecon command
490 tells SELinux to apply the labels to the actual content.
491 Now all new content created in these directories will automatically be
494 created with the correct label.
495
498 Container storage implements XFS project quota controls for overlay
499 storage containers and volumes. The directory used to store the con‐
500 tainers must be an XFS file system and be mounted with the pquota op‐
501 tion.
502 Example /etc/fstab entry:
505 /dev/podman/podman-var /var xfs defaults,x-systemd.device-timeout=0,pquota 1 2
508 Container storage generates project ids for each container and builtin
512 volume, but these project ids need to be unique for the XFS file sys‐
513 tem.
514 The xfs_quota tool can be used to assign a project id to the storage
517 driver directory, e.g.:
518 echo 100000:/var/lib/containers/storage/overlay >> /etc/projects
521 echo 200000:/var/lib/containers/storage/volumes >> /etc/projects
522 echo storage:100000 >> /etc/projid
523 echo volumes:200000 >> /etc/projid
524 xfs_quota -x -c 'project -s storage volumes' /<xfs mount point>
525 In the example above, the storage directory project id will be used as
529 a "start offset" and all containers will be assigned larger project ids
530 (e.g. >= 100000). Then the volumes directory project id will be used
531 as a "start offset" and all volumes will be assigned larger project ids
532 (e.g. >= 200000). This is a way to prevent xfs_quota management from
533 conflicting with containers/storage.
534
537 Distributions often provide a /usr/share/containers/storage.conf file
538 to define default storage configuration. Administrators can override
539 this file by creating /etc/containers/storage.conf to specify their own
540 configuration. Likewise rootless users can create a storage.conf file
541 to override the system storage.conf files. Files should be stored in
542 the $XDG_CONFIG_HOME/containers/storage.conf file. If $XDG_CONFIG_HOME
543 is not set then the file $HOME/.config/containers/storage.conf is used.
544 Note: The storage.conf file overrides all other storage.conf files.
547 Container engines run by users with a storage.conf file in their home
548 directory do not use options in the system storage.conf files.
549 /etc/projects - XFS persistent project root definition /etc/projid -
552 XFS project name mapping file
553
556 semanage(8), restorecon(8), mount(8), fuse-overlayfs(1), xfs_quota(8),
557 projects(5), projid(5)
558
561 May 2017, Originally compiled by Dan Walsh dwalsh@redhat.com
562 ⟨mailto:dwalsh@redhat.com⟩ Format copied from crio.conf man page cre‐
563 ated by Aleksa Sarai asarai@suse.de ⟨mailto:asarai@suse.de⟩
564Configuration Storagceontainers-storage.conf(5)(Container)