1crio.conf(5)(Kubernetes) Daemon crio.conf(5)(Kubernetes)
2Aleksa Sarai OCTOBER 2016
6
9 crio.conf - configuration file of the CRI-O OCI Kubernetes Container
10 Runtime daemon
11
15 The CRI-O configuration file specifies all of the available configura‐
16 tion options and command-line flags for the crio(8) OCI Kubernetes Con‐
17 tainer Runtime daemon ⟨./crio.8.md⟩, but in a TOML format that can be
18 more easily modified and versioned.
19 CRI-O supports partial configuration reload during runtime, which can
22 be done by sending SIGHUP to the running process. Currently supported
23 options in crio.conf are explicitly marked with 'This option supports
24 live configuration reload'.
25 The containers-registries.conf(5) file can be reloaded as well by send‐
28 ing SIGHUP to the crio process.
29 The default crio.conf is located at /etc/crio/crio.conf.
32
36 The TOML format ⟨https://github.com/toml-lang/toml⟩ is used as the en‐
37 coding of the configuration file. Every option and subtable listed here
38 is nested under a global "crio" table. No bare options are used. The
39 format of TOML can be simplified to:
40 [table]
43 option = value
44 [table.subtable1]
46 option = value
47 [table.subtable2]
49 option = value
50
54 CRI-O reads its storage defaults from the containers-storage.conf(5)
55 file located at /etc/containers/storage.conf. Modify this storage con‐
56 figuration if you want to change the system's defaults. If you want to
57 modify storage just for CRI-O, you can change the storage configuration
58 options here.
59 root="/var/lib/containers/storage"
62 Path to the "root directory". CRI-O stores all of its data, including
63 containers images, in this directory.
64 runroot="/var/run/containers/storage"
67 Path to the "run directory". CRI-O stores all of its state in this
68 directory.
69 storage_driver="overlay"
72 Storage driver used to manage the storage of images and containers.
73 Please refer to containers-storage.conf(5) to see all available storage
74 drivers.
75 storage_option=[]
78 List to pass options to the storage driver. Please refer to contain‐
79 ers-storage.conf(5) to see all available storage options.
80 log_dir="/var/log/crio/pods"
83 The default log directory where all logs will go unless directly
84 specified by the kubelet. The log directory specified must be an abso‐
85 lute directory.
86 version_file="/var/run/crio/version"
89 Location for CRI-O to lay down the temporary version file.
90 It is used to check if crio wipe should wipe containers, which should
91 always happen on a node reboot
92 version_file_persist="/var/lib/crio/version"
95 Location for CRI-O to lay down the persistent version file.
96 It is used to check if crio wipe should wipe images, which should
97 only happen when CRI-O has been upgraded
98 internal_wipe=true
101 This option is currently DEPRECATED, and will be removed in the fu‐
102 ture.
103 Whether CRI-O should wipe containers after a reboot and images after
104 an upgrade when the server starts.
105 If set to false, one must run crio wipe to wipe the containers and
106 images in these situations.
107 clean_shutdown_file="/var/lib/crio/clean.shutdown"
110 Location for CRI-O to lay down the clean shutdown file.
111 It is used to check whether crio had time to sync before shutting
112 down.
113 If not found, crio wipe will clear the storage directory.
114
117 The crio.api table contains settings for the kubelet/gRPC interface.
118 listen="/var/run/crio/crio.sock"
121 Path to AF_LOCAL socket on which CRI-O will listen.
122 stream_address="127.0.0.1"
125 IP address on which the stream server will listen.
126 stream_port="0"
129 The port on which the stream server will listen. If the port is set
130 to "0", then CRI-O will allocate a random free port number.
131 stream_enable_tls=false
134 Enable encrypted TLS transport of the stream server.
135 stream_idle_timeout=""
138 Length of time until open streams terminate due to lack of activity.
139 stream_tls_cert=""
142 Path to the x509 certificate file used to serve the encrypted stream.
143 This file can change and CRI-O will automatically pick up the changes
144 within 5 minutes.
145 stream_tls_key=""
148 Path to the key file used to serve the encrypted stream. This file
149 can change and CRI-O will automatically pick up the changes within 5
150 minutes.
151 stream_tls_ca=""
154 Path to the x509 CA(s) file used to verify and authenticate client
155 communication with the encrypted stream. This file can change and CRI-O
156 will automatically pick up the changes within 5 minutes.
157 grpc_max_send_msg_size=83886080
160 Maximum grpc send message size in bytes. If not set or <=0, then CRI-
161 O will default to 80 * 1024 * 1024.
162 grpc_max_recv_msg_size=83886080
165 Maximum grpc receive message size. If not set or <= 0, then CRI-O
166 will default to 80 * 1024 * 1024.
167
170 The crio.runtime table contains settings pertaining to the OCI runtime
171 used and options for how to set up and manage the OCI runtime.
172 default_runtime="runc"
175 The name of the OCI runtime to be used as the default.
176 default_ulimits=[]
179 A list of ulimits to be set in containers by default, specified as
180 "=:", for example:"nofile=1024:2048". If nothing is set here, settings
181 will be inherited from the CRI-O daemon.
182 no_pivot=false
185 If true, the runtime will not use pivot_root, but instead use
186 MS_MOVE.
187 decryption_keys_path="/etc/crio/keys/"
190 Path where the keys required for image decryption are located
191 conmon=""
194 Path to the conmon binary, used for monitoring the OCI runtime. Will
195 be searched for using $PATH if empty.
196 This option is currently deprecated, and will be replaced with Run‐
197 timeHandler.MonitorPath.
198 conmon_cgroup=""
201 Cgroup setting for conmon
202 This option is currently deprecated, and will be replaced with Run‐
203 timeHandler.MonitorCgroup.
204 conmon_env=[]
207 Environment variable list for the conmon process, used for passing
208 necessary environment variables to conmon or the runtime.
209 This option is currently deprecated, and will be replaced with Run‐
210 timeHandler.MonitorEnv.
211 default_env=[]
214 Additional environment variables to set for all the containers. These
215 are overridden if set in the container image spec or in the container
216 runtime configuration.
217 selinux=false
220 If true, SELinux will be used for pod separation on the host.
221 seccomp_profile=""
224 Path to the seccomp.json profile which is used as the default seccomp
225 profile for the runtime. If not specified, then the internal default
226 seccomp profile will be used.
227 seccomp_use_default_when_empty=true
230 Changes the meaning of an empty seccomp profile. By default (and ac‐
231 cording to CRI spec), an empty profile means unconfined.
232 This option tells CRI-O to treat an empty profile as the default pro‐
233 file, which might increase security.
234 apparmor_profile=""
237 Used to change the name of the default AppArmor profile of CRI-O. The
238 default profile name is "crio-default".
239 blockio_config_file=""
242 Path to the blockio class configuration file for configuring the
243 cgroup blockio controller.
244 cdi_spec_dirs=[]
247 Directories to scan for Container Device Interface Specifications to
248 enable CDI device injection. For more details about CDI and the syntax
249 of CDI Spec files please refer to https://github.com/container-orches‐
250 trated-devices/container-device-interface.
251 Directories later in the list have precedence over earlier ones. The
254 default directory list is:
255 cdi_spec_dirs = [
258 "/etc/cdi",
259 "/var/run/cdi",
260 ]
261 irqbalance_config_file="/etc/sysconfig/irqbalance"
265 Used to change irqbalance service config file which is used by CRI-O.
266 For CentOS/SUSE, this file is located at /etc/sysconfig/irqbalance.
267 For Ubuntu, this file is located at /etc/default/irqbalance.
268 rdt_config_file=""
271 Path to the RDT configuration file for configuring the resctrl
272 pseudo-filesystem.
273 cgroup_manager="systemd"
276 Cgroup management implementation used for the runtime.
277 default_capabilities=[]
280 List of default capabilities for containers. If it is empty or com‐
281 mented out, only the capabilities defined in the container json file by
282 the user/kube will be added.
283 The default list is:
286 default_capabilities = [
289 "CHOWN",
290 "DAC_OVERRIDE",
291 "FSETID",
292 "FOWNER",
293 "SETGID",
294 "SETUID",
295 "SETPCAP",
296 "NET_BIND_SERVICE",
297 "KILL",
298 ]
299 default_sysctls=[]
303 List of default sysctls. If it is empty or commented out, only the
304 sysctls defined in the container json file by the user/kube will be
305 added.
306 One example would be allowing ping inside of containers. On systems
309 that support /proc/sys/net/ipv4/ping_group_range, the default list
310 could be:
311 default_sysctls = [
314 "net.ipv4.ping_group_range = 0 2147483647",
315 ]
316 allowed_devices=[]
320 List of devices on the host that a user can specify with the "io.ku‐
321 bernetes.cri-o.Devices" allowed annotation.
322 additional_devices=[]
325 List of additional devices. Specified as "::", for example: "--addi‐
326 tional-devices=/dev/sdc:/dev/xvdc:rwm". If it is empty or commented
327 out, only the devices defined in the container json file by the
328 user/kube will be added.
329 hooks_dir=["path", ...]
332 Each *.json file in the path configures a hook for CRI-O containers.
333 For more details on the syntax of the JSON files and the semantics of
334 hook injection, see oci-hooks(5). CRI-O currently support both the
335 1.0.0 and 0.1.0 hook schemas, although the 0.1.0 schema is deprecated.
336 Paths listed later in the array have higher precedence (oci-hooks(5)
339 discusses directory precedence).
340 For the annotation conditions, CRI-O uses the Kubernetes annotations,
343 which are a subset of the annotations passed to the OCI runtime. For
344 example, io.kubernetes.cri-o.Volumes is part of the OCI runtime config‐
345 uration annotations, but it is not part of the Kubernetes annotations
346 being matched for hooks.
347 For the bind-mount conditions, only mounts explicitly requested by Ku‐
350 bernetes configuration are considered. Bind mounts that CRI-O inserts
351 by default (e.g. /dev/shm) are not considered.
352 default_mounts=[]
355 List of default mounts for each container. Deprecated: this option
356 will be removed in future versions in favor of default_mounts_file.
357 default_mounts_file=""
360 Path to the file specifying the defaults mounts for each container.
361 The format of the config is /SRC:/DST, one mount per line. Notice that
362 CRI-O reads its default mounts from the following two files:
363 1) `/etc/containers/mounts.conf` (i.e., default_mounts_file): This is the override file, where users can either add in their own default mounts, or override the default mounts shipped with the package.
366 2) `/usr/share/containers/mounts.conf`: This is the default file read for mounts. If you want CRI-O to read from a different, specific mounts file, you can change the default_mounts_file. Note, if this is done, CRI-O will only add mounts it finds in this file.
368 pids_limit=0
372 Maximum number of processes allowed in a container.
373 This option is deprecated. The Kubelet flag --pod-pids-limit should
374 be used instead.
375 log_filter=""
378 Filter the log messages by the provided regular expression. This op‐
379 tion supports live configuration reload. For example 'request:.*' fil‐
380 ters all gRPC requests.
381 log_level="info"
384 Changes the verbosity of the logs based on the level it is set to.
385 Options are fatal, panic, error, warn, info, debug, and trace. This op‐
386 tion supports live configuration reload.
387 log_size_max=-1
390 Maximum size allowed for the container log file. Negative numbers in‐
391 dicate that no size limit is imposed. If it is positive, it must be >=
392 8192 to match/exceed conmon's read buffer. The file is truncated and
393 re-opened so the limit is never exceeded.
394 This option is deprecated. The Kubelet flag --container-log-max-size
395 should be used instead.
396 log_to_journald=false
399 Whether container output should be logged to journald in addition to
400 the kuberentes log file.
401 container_exits_dir="/var/run/crio/exits"
404 Path to directory in which container exit files are written to by
405 conmon.
406 container_attach_socket_dir="/var/run/crio"
409 Path to directory for container attach sockets.
410 bind_mount_prefix=""
413 A prefix to use for the source of the bind mounts. This option would
414 be useful when running CRI-O in a container and the / directory on the
415 host is mounted as /host in the container. Then if CRI-O runs with the
416 --bind-mount-prefix=/host option, CRI-O would add the /host directory
417 to any bind mounts it hands over to CRI. If Kubernetes asked to have
418 /var/lib/foobar bind mounted into the container, then CRI-O would bind
419 mount /host/var/lib/foobar. Since CRI-O itself is running in a con‐
420 tainer with / or the host mounted on /host, the container would end up
421 with /var/lib/foobar from the host mounted in the container rather than
422 /var/lib/foobar from the CRI-O container.
423 read_only=false
426 If set to true, all containers will run in read-only mode.
427 uid_mappings=""
430 The UID mappings for the user namespace of each container. A range is
431 specified in the form containerUID:HostUID:Size. Multiple ranges must
432 be separated by comma.
433 minimum_mappable_uid=-1
436 The lowest host UID which can be specified in mappings supplied, ei‐
437 ther as part of a uid_mappings or as part of a request received over
438 CRI, for a pod that will be run as a UID other than 0.
439 gid_mappings=""
442 The GID mappings for the user namespace of each container. A range is
443 specified in the form containerGID:HostGID:Size. Multiple ranges must
444 be separated by comma.
445 minimum_mappable_gid=-1
448 The lowest host GID which can be specified in mappings supplied, ei‐
449 ther as part of a gid_mappings or as part of a request received over
450 CRI, for a pod that will be run as a UID other than 0.
451 ctr_stop_timeout=30
454 The minimal amount of time in seconds to wait before issuing a time‐
455 out regarding the proper termination of the container.
456 drop_infra_ctr=true
459 Determines whether we drop the infra container when a pod does not
460 have a private PID namespace, and does not use a kernel separating run‐
461 time (like kata).
462 Requies manage_ns_lifecycle to be true.
463 infra_ctr_cpuset=""
466 Determines the CPU set to run infra containers. If not specified,
467 the CRI-O will use all online CPUs to run infra containers.
468 You can specify CPUs in the Linux CPU list format.
469 To get better isolation for guaranteed pods, set this parameter to
470 be equal to kubelet reserved-cpus.
471 namespaces_dir="/var/run"
474 The directory where the state of the managed namespaces gets tracked.
475 Only used when manage_ns_lifecycle is true
476 pinns_path=""
479 The path to find the pinns binary, which is needed to manage name‐
480 space lifecycle
481 absent_mount_sources_to_reject=[]
484 A list of paths that, when absent from the host, will cause a con‐
485 tainer creation to fail (as opposed to the current behavior of creating
486 a directory).
487 device_ownership_from_security_context=false
490 Changes the default behavior of setting container devices uid/gid
491 from CRI's SecurityContext (RunAsUser/RunAsGroup) instead of taking
492 host's uid/gid.
493 CRIO.RUNTIME.RUNTIMES TABLE
496 The "crio.runtime.runtimes" table defines a list of OCI compatible run‐
497 times. The runtime to use is picked based on the runtime handler pro‐
498 vided by the CRI. If no runtime handler is provided, the runtime will
499 be picked based on the level of trust of the workload.
500 runtime_path=""
503 Path to the OCI compatible runtime used for this runtime handler.
504 runtime_root=""
507 Root directory used to store runtime data
508 runtime_type="oci"
511 Type of the runtime used for this runtime handler. "oci", "vm"
512 runtime_config_path=""
515 Path to the runtime configuration file, should only be used with VM
516 runtime types
517 privileged_without_host_devices=false
520 Whether this runtime handler prevents host devices from being passed
521 to privileged containers.
522 allowed_annotations=[]
525 This field is currently DEPRECATED. If you'd like to use allowed_an‐
526 notations, please use a workload.
527 A list of experimental annotations this runtime handler is allowed to
528 process.
529 The currently recognized values are:
530 "io.kubernetes.cri-o.userns-mode" for configuring a user namespace
531 for the pod.
532 "io.kubernetes.cri-o.Devices" for configuring devices for the pod.
533 "io.kubernetes.cri-o.ShmSize" for configuring the size of /dev/shm.
534 "io.kubernetes.cri-o.UnifiedCgroup.$CTR_NAME" for configuring the
535 cgroup v2 unified block for a container.
536 "io.containers.trace-syscall" for tracing syscalls via the OCI sec‐
537 comp BPF hook.
538 CRIO.RUNTIME.WORKLOADS TABLE
541 The "crio.runtime.workloads" table defines a list of workloads - a way
542 to customize the behavior of a pod and container. A workload is chosen
543 for a pod based on whether the workload's activation_annotation is an
544 annotation on the pod.
545 activation_annotation=""
548 activation_annotation is the pod annotation that activates these
549 workload settings.
550 annotation_prefix=""
553 annotation_prefix is the way a pod can override a specific resource
554 for a container.
555 The full annotation must be of the form $annotation_prefix.$re‐
556 source/$ctrname = $value.
557 allowed_annotations=[]
560 allowed_annotations is a slice of experimental annotations that this
561 workload is allowed to process.
562 The currently recognized values are:
563 "io.kubernetes.cri-o.userns-mode" for configuring a user namespace
564 for the pod.
565 "io.kubernetes.cri-o.Devices" for configuring devices for the pod.
566 "io.kubernetes.cri-o.ShmSize" for configuring the size of /dev/shm.
567 "io.kubernetes.cri-o.UnifiedCgroup.$CTR_NAME" for configuring the
568 cgroup v2 unified block for a container.
569 "io.containers.trace-syscall" for tracing syscalls via the OCI sec‐
570 comp BPF hook.
571 CRIO.RUNTIME.WORKLOAD.RESOURCES TABLE
574 The resources table is a structure for overriding certain resources for
575 pods using this workload. This structure provides a default value, and
576 can be overridden by using the AnnotationPrefix.
577 cpushares="" Specifies the number of CPU shares this pod has access to.
580 cpuset="" Specifies the cpuset this pod has access to.
583
586 The crio.image table contains settings pertaining to the management of
587 OCI images.
588 CRI-O reads its configured registries defaults from the system wide
591 containers-registries.conf(5) located in /etc/containers/reg‐
592 istries.conf. If you want to modify just CRI-O, you can change the reg‐
593 istries configuration in this file. Otherwise, leave insecure_reg‐
594 istries and registries commented out to use the system's defaults from
595 /etc/containers/registries.conf.
596 default_transport="docker://"
599 Default transport for pulling images from a remote container storage.
600 global_auth_file=""
603 The path to a file like /var/lib/kubelet/config.json holding creden‐
604 tials necessary for pulling images from secure registries.
605 pause_image="registry.k8s.io/pause:3.6"
608 The image used to instantiate infra containers. This option supports
609 live configuration reload.
610 pause_image_auth_file=""
613 The path to a file like /var/lib/kubelet/config.json holding creden‐
614 tials specific to pulling the pause_image from above. This option sup‐
615 ports live configuration reload.
616 pause_command="/pause"
619 The command to run to have a container stay in the paused state. This
620 option supports live configuration reload.
621 signature_policy=""
624 Path to the file which decides what sort of policy we use when decid‐
625 ing whether or not to trust an image that we've pulled. It is not rec‐
626 ommended that this option be used, as the default behavior of using the
627 system-wide default policy (i.e., /etc/containers/policy.json) is most
628 often preferred. Please refer to containers-policy.json(5) for more de‐
629 tails.
630 image_volumes="mkdir"
633 Controls how image volumes are handled. The valid values are mkdir,
634 bind and ignore; the latter will ignore volumes entirely.
635 insecure_registries=[]
638 List of registries to skip TLS verification for pulling images.
639 registries=["docker.io"]
642 List of registries to be used when pulling an unqualified image. Note
643 support for this option has been dropped and it has no effect. Please
644 refer to containers-registries.conf(5) for configuring unqualified-
645 search registries.
646 big_files_temporary_dir=""
649 Path to the temporary directory to use for storing big files, used to
650 store image blobs and data streams related to containers image manage‐
651 ment.
652 separate_pull_cgroup=""
655 [EXPERIMENTAL] If its value is set, then images are pulled into the
656 specified cgroup. If its value is set to "pod", then the pod's cgroup
657 is used. It is currently supported only with the systemd cgroup man‐
658 ager.
659
662 The crio.network table containers settings pertaining to the management
663 of CNI plugins.
664 cni_default_network=""
667 The default CNI network name to be selected. If not set or "", then
668 CRI-O will pick-up the first one found in network_dir.
669 network_dir="/etc/cni/net.d/"
672 Path to the directory where CNI configuration files are located.
673 plugin_dirs=["/opt/cni/bin/",]
676 List of paths to directories where CNI plugin binaries are located.
677
680 The crio.metrics table containers settings pertaining to the Prometheus
681 based metrics retrieval.
682 enable_metrics=false
685 Globally enable or disable metrics support.
686 enable_custom_shm_size=false If set to true, enable users to set a cus‐
689 tom shm size instead of using the default value of 64M. The shm size
690 can be set through K8S annotation with the key "io.kubernetes.cri-
691 o.ShmSize", and the value representing the size in human readable for‐
692 mat. For example: "io.kubernetes.cri-o.ShmSize: 128Mi"
693 metrics_collectors=["operations", "operations_latency_microseconds_to‐
696 tal", "operations_latency_microseconds", "operations_errors", "im‐
697 age_pulls_by_digest", "image_pulls_by_name", "im‐
698 age_pulls_by_name_skipped", "image_pulls_failures", "image_pulls_suc‐
699 cesses", "image_pulls_layer_size", "image_layer_reuse", "contain‐
700 ers_oom_total", "containers_oom", "processes_defunct"]
701 Enabled metrics collectors
702 metrics_port=9090
705 The port on which the metrics server will listen.
706 metrics_socket=""
709 The socket on which the metrics server will listen.
710 metrics_cert=""
713 The certificate for the secure metrics server.
714 metrics_key=""
717 The certificate key for the secure metrics server.
718
721 [EXPERIMENTAL] The crio.tracing table containers settings pertaining to
722 the export of OpenTelemetry trace data.
723 enable_tracing=false
726 Globally enable or disable OpenTelemetry trace data exporting.
727 tracing_endpoint="0.0.0.0:4317"
730 Address on which the gRPC trace collector will listen.
731 tracing_sampling_rate_per_million=""
734 Number of samples to collect per million OpenTelemetry spans.
735
738 The crio.stats table specifies all necessary configuration for report‐
739 ing container and pod stats.
740 stats_collection_period=0
743 The number of seconds between collecting pod and container stats. If
744 set to 0, the stats are collected on-demand instead.
745
749 crio.conf.d(5), containers-storage.conf(5), containers-policy.json(5),
750 containers-registries.conf(5), crio(8)
751
755 Aug 2018, Update to the latest state by Valentin Rothberg vroth‐
756 berg@suse.com ⟨mailto:vrothberg@suse.com⟩
757 Oct 2016, Originally compiled by Aleksa Sarai asarai@suse.de
760 ⟨mailto:asarai@suse.de⟩
761Runtime Container crio.conf(5)(Kubernetes)