1crio(8) System Manager's Manual crio(8)
2
6 crio - OCI-based implementation of Kubernetes Container Runtime Inter‐
7 face
8
12 crio
13 [--absent-mount-sources-to-reject]=[value]
16 [--additional-devices]=[value]
17 [--allowed-devices]=[value]
18 [--apparmor-profile]=[value]
19 [--big-files-temporary-dir]=[value]
20 [--bind-mount-prefix]=[value]
21 [--blockio-config-file]=[value]
22 [--cdi-spec-dirs]=[value]
23 [--cgroup-manager]=[value]
24 [--clean-shutdown-file]=[value]
25 [--cni-config-dir]=[value]
26 [--cni-default-network]=[value]
27 [--cni-plugin-dir]=[value]
28 [--config-dir|-d]=[value]
29 [--config|-c]=[value]
30 [--conmon-cgroup]=[value]
31 [--conmon-env]=[value]
32 [--conmon]=[value]
33 [--container-attach-socket-dir]=[value]
34 [--container-exits-dir]=[value]
35 [--ctr-stop-timeout]=[value]
36 [--decryption-keys-path]=[value]
37 [--default-capabilities]=[value]
38 [--default-env]=[value]
39 [--default-mounts-file]=[value]
40 [--default-runtime]=[value]
41 [--default-sysctls]=[value]
42 [--default-transport]=[value]
43 [--default-ulimits]=[value]
44 [--device-ownership-from-security-context]
45 [--drop-infra-ctr]
46 [--enable-metrics]
47 [--enable-profile-unix-socket]
48 [--enable-tracing]
49 [--gid-mappings]=[value]
50 [--global-auth-file]=[value]
51 [--grpc-max-recv-msg-size]=[value]
52 [--grpc-max-send-msg-size]=[value]
53 [--help|-h]
54 [--hooks-dir]=[value]
55 [--image-volumes]=[value]
56 [--infra-ctr-cpuset]=[value]
57 [--insecure-registry]=[value]
58 [--internal-wipe]
59 [--irqbalance-config-file]=[value]
60 [--listen]=[value]
61 [--log-dir]=[value]
62 [--log-filter]=[value]
63 [--log-format]=[value]
64 [--log-journald]
65 [--log-level|-l]=[value]
66 [--log-size-max]=[value]
67 [--log]=[value]
68 [--metrics-cert]=[value]
69 [--metrics-collectors]=[value]
70 [--metrics-key]=[value]
71 [--metrics-port]=[value]
72 [--metrics-socket]=[value]
73 [--minimum-mappable-gid]=[value]
74 [--minimum-mappable-uid]=[value]
75 [--namespaces-dir]=[value]
76 [--no-pivot]
77 [--pause-command]=[value]
78 [--pause-image-auth-file]=[value]
79 [--pause-image]=[value]
80 [--pids-limit]=[value]
81 [--pinns-path]=[value]
82 [--profile-cpu]=[value]
83 [--profile-mem]=[value]
84 [--profile-port]=[value]
85 [--profile]
86 [--rdt-config-file]=[value]
87 [--read-only]
88 [--registry]=[value]
89 [--root|-r]=[value]
90 [--runroot]=[value]
91 [--runtimes]=[value]
92 [--seccomp-profile]=[value]
93 [--seccomp-use-default-when-empty]
94 [--selinux]
95 [--separate-pull-cgroup]=[value]
96 [--signature-policy]=[value]
97 [--stats-collection-period]=[value]
98 [--storage-driver|-s]=[value]
99 [--storage-opt]=[value]
100 [--stream-address]=[value]
101 [--stream-enable-tls]
102 [--stream-idle-timeout]=[value]
103 [--stream-port]=[value]
104 [--stream-tls-ca]=[value]
105 [--stream-tls-cert]=[value]
106 [--stream-tls-key]=[value]
107 [--tracing-endpoint]=[value]
108 [--tracing-sampling-rate-per-million]=[value]
109 [--uid-mappings]=[value]
110 [--version-file-persist]=[value]
111 [--version-file]=[value]
112 [--version|-v]
113
118 OCI-based implementation of Kubernetes Container Runtime Interface Dae‐
119 mon
120 crio is meant to provide an integration path between OCI conformant
123 runtimes and the kubelet. Specifically, it implements the Kubelet Con‐
124 tainer Runtime Interface (CRI) using OCI conformant runtimes. The scope
125 of crio is tied to the scope of the CRI.
126 1. Support multiple image formats including the existing
129 Docker and OCI image formats.
130 2. Support for multiple means to download images including
132 trust & image verification.
133 3. Container image management (managing image layers, overlay
135 filesystems, etc).
136 4. Container process lifecycle management.
138 5. Monitoring and logging required to satisfy the CRI.
140 6. Resource isolation as required by the CRI.
142 Usage:
146 crio [GLOBAL OPTIONS] command [COMMAND OPTIONS] [ARGUMENTS...]
149
154 --absent-mount-sources-to-reject="": A list of paths that, when absent
155 from the host, will cause a container creation to fail (as opposed to
156 the current behavior of creating a directory). (default: [])
157 --additional-devices="": Devices to add to the containers (default:
160 [])
161 --allowed-devices="": Devices a user is allowed to specify with the
164 "io.kubernetes.cri-o.Devices" allowed annotation (default: [/dev/fuse])
165 --apparmor-profile="": Name of the apparmor profile to be used as the
168 runtime's default. This only takes effect if the user does not specify
169 a profile via the Kubernetes Pod's metadata annotation. (default: crio-
170 default)
171 --big-files-temporary-dir="": Path to the temporary directory to use
174 for storing big files, used to store image blobs and data streams re‐
175 lated to containers image management.
176 --bind-mount-prefix="": A prefix to use for the source of the bind
179 mounts. This option would be useful if you were running CRI-O in a con‐
180 tainer. And had / mounted on /host in your container. Then if you ran
181 CRI-O with the --bind-mount-prefix=/host option, CRI-O would add /host
182 to any bind mounts it is handed over CRI. If Kubernetes asked to have
183 /var/lib/foobar bind mounted into the container, then CRI-O would bind
184 mount /host/var/lib/foobar. Since CRI-O itself is running in a con‐
185 tainer with / or the host mounted on /host, the container would end up
186 with /var/lib/foobar from the host mounted in the container rather then
187 /var/lib/foobar from the CRI-O container. (default: "")
188 --blockio-config-file="": Path to the blockio class configuration file
191 for configuring the cgroup blockio controller.
192 --cdi-spec-dirs="": Directories to scan for CDI Spec files (default:
195 [/etc/cdi /var/run/cdi])
196 --cgroup-manager="": cgroup manager (cgroupfs or systemd) (default:
199 systemd)
200 --clean-shutdown-file="": Location for CRI-O to lay down the clean
203 shutdown file. It indicates whether we've had time to sync changes to
204 disk before shutting down. If not found, crio wipe will clear the stor‐
205 age directory (default: /var/lib/crio/clean.shutdown)
206 --cni-config-dir="": CNI configuration files directory (default:
209 /etc/cni/net.d/)
210 --cni-default-network="": Name of the default CNI network to select. If
213 not set or "", then CRI-O will pick-up the first one found in --cni-
214 config-dir.
215 --cni-plugin-dir="": CNI plugin binaries directory (default: [])
218 --config, -c="": Path to configuration file (default:
221 /etc/crio/crio.conf)
222 --config-dir, -d="": Path to the configuration drop-in directory.
225 This directory will be recursively iterated and each file gets ap‐
226 plied
227 to the configuration in their processing order. This means that a
228 configuration file named '00-default' has a lower priority than a
229 file
230 named '01-my-overwrite'.
231 The global config file, provided via '--config,-c' or per default
232 in
233 /etc/crio/crio.conf, always has a lower priority than the files in
234 the directory specified
235 by '--config-dir,-d'.
236 Besides that, provided command line parameters have a higher prior‐
237 ity
238 than any configuration file. (default: /etc/crio/crio.conf.d)
239 --conmon="": Path to the conmon binary, used for monitoring the OCI
242 runtime. Will be searched for using $PATH if empty. This option is dep‐
243 recated, and will be removed in the future. (default: "")
244 --conmon-cgroup="": cgroup to be used for conmon process. This option
247 is deprecated and will be removed in the future.
248 --conmon-env="": Environment variable list for the conmon process, used
251 for passing necessary environment variables to conmon or the runtime.
252 This option is deprecated and will be removed in the future. (default:
253 [])
254 --container-attach-socket-dir="": Path to directory for container at‐
257 tach sockets (default: /var/run/crio)
258 --container-exits-dir="": Path to directory in which container exit
261 files are written to by conmon (default: /var/run/crio/exits)
262 --ctr-stop-timeout="": The minimal amount of time in seconds to wait
265 before issuing a timeout regarding the proper termination of the con‐
266 tainer. The lowest possible value is 30s, whereas lower values are not
267 considered by CRI-O (default: 30)
268 --decryption-keys-path="": Path to load keys for image decryption. (de‐
271 fault: /etc/crio/keys/)
272 --default-capabilities="": Capabilities to add to the containers (de‐
275 fault: [CHOWN DAC_OVERRIDE FSETID FOWNER SETGID SETUID SETPCAP
276 NET_BIND_SERVICE KILL])
277 --default-env="": Additional environment variables to set for all con‐
280 tainers (default: [])
281 --default-mounts-file="": Path to default mounts file (default: "")
284 --default-runtime="": Default OCI runtime from the runtimes config (de‐
287 fault: runc)
288 --default-sysctls="": Sysctls to add to the containers (default: [])
291 --default-transport="": A prefix to prepend to image names that cannot
294 be pulled as-is (default: docker://)
295 --default-ulimits="": Ulimits to apply to containers by default
298 (name=soft:hard) (default: []) (default: [])
299 --device-ownership-from-security-context: Set devices' uid/gid owner‐
302 ship from runAsUser/runAsGroup
303 --drop-infra-ctr: Determines whether pods are created without an infra
306 container, when the pod is not using a pod level PID namespace (de‐
307 fault: true)
308 --enable-metrics: Enable metrics endpoint for the server on local‐
311 host:9090
312 --enable-profile-unix-socket: Enable pprof profiler on crio unix domain
315 socket
316 --enable-tracing: Enable OpenTelemetry trace data exporting
319 --gid-mappings="": Specify the GID mappings to use for the user name‐
322 space (default: "")
323 --global-auth-file="": Path to a file like /var/lib/kubelet/config.json
326 holding credentials necessary for pulling images from secure registries
327 (default: "")
328 --grpc-max-recv-msg-size="": Maximum grpc receive message size in bytes
331 (default: 83886080)
332 --grpc-max-send-msg-size="": Maximum grpc receive message size (de‐
335 fault: 83886080)
336 --help, -h: show help
339 --hooks-dir="": Set the OCI hooks directory path (may be set multiple
342 times)
343 If one of the directories does not exist, then CRI-O will automati‐
344 cally
345 skip them.
346 Each '*.json' file in the path configures a hook for CRI-O
347 containers. For more details on the syntax of the JSON files and
348 the semantics of hook injection, see 'oci-hooks(5)'. CRI-O
349 currently support both the 1.0.0 and 0.1.0 hook schemas, although
350 the 0.1.0 schema is deprecated.
351 This option may be set multiple times; paths from later options
352 have higher precedence ('oci-hooks(5)' discusses directory
353 precedence).
354 For the annotation conditions, CRI-O uses the Kubernetes
355 annotations, which are a subset of the annotations passed to the
356 OCI runtime. For example, 'io.kubernetes.cri-o.Volumes' is part of
357 the OCI runtime configuration annotations, but it is not part of
358 the Kubernetes annotations being matched for hooks.
359 For the bind-mount conditions, only mounts explicitly requested by
360 Kubernetes configuration are considered. Bind mounts that CRI-O
361 inserts by default (e.g. '/dev/shm') are not considered. (default:
362 [/usr/share/containers/oci/hooks.d])
363 --image-volumes="": Image volume handling ('mkdir', 'bind', or 'ig‐
366 nore')
367 1. mkdir: A directory is created inside the container root filesys‐
368 tem for
369 the volumes.
370 2. bind: A directory is created inside container state directory
371 and bind
372 mounted into the container for the volumes. 3. ignore: All
373 volumes are just ignored and no action is taken. (default: mkdir)
374 --infra-ctr-cpuset="": CPU set to run infra containers, if not speci‐
377 fied CRI-O will use all online CPUs to run infra containers (default:
378 '').
379 --insecure-registry="": Enable insecure registry communication, i.e.,
382 enable un-encrypted and/or untrusted communication.
383 1. List of insecure registries can contain an element with CIDR no‐
384 tation to
385 specify a whole subnet.
386 2. Insecure registries accept HTTP or accept HTTPS with certifi‐
387 cates from
388 unknown CAs.
389 3. Enabling '--insecure-registry' is useful when running a local
390 registry.
391 However, because its use creates security vulnerabilities, it
392 should ONLY
393 be enabled for testing purposes. For increased security, users
394 should add
395 their CA to their system's list of trusted CAs instead of using
396 '--insecure-registry'. (default: [])
397 --internal-wipe: Whether CRI-O should wipe containers after a reboot
400 and images after an upgrade when the server starts. If set to false,
401 one must run crio wipe to wipe the containers and images in these situ‐
402 ations. This option is deprecated, and will be removed in the future.
403 --irqbalance-config-file="": The irqbalance service config file which
406 is used by CRI-O. (default: /etc/sysconfig/irqbalance)
407 --listen="": Path to the CRI-O socket (default:
410 /var/run/crio/crio.sock)
411 --log="": Set the log file path where internal debug information is
414 written
415 --log-dir="": Default log directory where all logs will go unless di‐
418 rectly specified by the kubelet (default: /var/log/crio/pods)
419 --log-filter="": Filter the log messages by the provided regular ex‐
422 pression. For example 'request.*' filters all gRPC requests.
423 --log-format="": Set the format used by logs: 'text' or 'json' (de‐
426 fault: text)
427 --log-journald: Log to systemd journal (journald) in addition to kuber‐
430 netes log file (default: false)
431 --log-level, -l="": Log messages above specified level: trace, debug,
434 info, warn, error, fatal or panic (default: info)
435 --log-size-max="": Maximum log size in bytes for a container. If it is
438 positive, it must be >= 8192 to match/exceed conmon read buffer. This
439 option is deprecated. The Kubelet flag '--container-log-max-size'
440 should be used instead. (default: -1)
441 --metrics-cert="": Certificate for the secure metrics endpoint
444 --metrics-collectors="": Enabled metrics collectors (default: [opera‐
447 tions operations_latency_microseconds_total operations_latency_mi‐
448 croseconds operations_errors image_pulls_by_digest image_pulls_by_name
449 image_pulls_by_name_skipped image_pulls_failures image_pulls_successes
450 image_pulls_layer_size image_layer_reuse containers_oom_total contain‐
451 ers_oom processes_defunct operations_total operations_latency_seconds
452 operations_latency_seconds_total operations_errors_total im‐
453 age_pulls_bytes_total image_pulls_skipped_bytes_total image_pulls_fail‐
454 ure_total image_pulls_success_total image_layer_reuse_total contain‐
455 ers_oom_count_total])
456 --metrics-key="": Certificate key for the secure metrics endpoint
459 --metrics-port="": Port for the metrics endpoint (default: 9090)
462 --metrics-socket="": Socket for the metrics endpoint
465 --minimum-mappable-gid="": Specify the lowest host GID which can be
468 specified in mappings for a pod that will be run as a UID other than 0
469 (default: -1)
470 --minimum-mappable-uid="": Specify the lowest host UID which can be
473 specified in mappings for a pod that will be run as a UID other than 0
474 (default: -1)
475 --namespaces-dir="": The directory where the state of the managed name‐
478 spaces gets tracked. Only used when manage-ns-lifecycle is true (de‐
479 fault: /var/run)
480 --no-pivot: If true, the runtime will not use pivot_root, but instead
483 use MS_MOVE (default: false)
484 --pause-command="": Path to the pause executable in the pause image
487 (default: /pause)
488 --pause-image="": Image which contains the pause executable (default:
491 registry.k8s.io/pause:3.6)
492 --pause-image-auth-file="": Path to a config file containing creden‐
495 tials for --pause-image (default: "")
496 --pids-limit="": Maximum number of processes allowed in a container.
499 This option is deprecated. The Kubelet flag '--pod-pids-limit' should
500 be used instead. (default: 0)
501 --pinns-path="": The path to find the pinns binary, which is needed to
504 manage namespace lifecycle. Will be searched for in $PATH if empty (de‐
505 fault: "")
506 --profile: Enable pprof remote profiler on localhost:6060
509 --profile-cpu="": Write a pprof CPU profile to the provided path
512 --profile-mem="": Write a pprof memory profile to the provided path
515 --profile-port="": Port for the pprof profiler (default: 6060)
518 --rdt-config-file="": Path to the RDT configuration file for configur‐
521 ing the resctrl pseudo-filesystem
522 --read-only: Setup all unprivileged containers to run as read-only. Au‐
525 tomatically mounts tmpfs on /run, /tmp and /var/tmp. (default: false)
526 --registry="": Registry to be prepended when pulling unqualified im‐
529 ages, can be specified multiple times (default: [])
530 --root, -r="": The CRI-O root directory (default: /var/lib/contain‐
533 ers/storage)
534 --runroot="": The CRI-O state directory (default: /run/containers/stor‐
537 age)
538 --runtimes="": OCI runtimes, format is runtime_name:runtime_path:run‐
541 time_root:runtime_type:privileged_without_host_devices:runtime_con‐
542 fig_path (default: [])
543 --seccomp-profile="": Path to the seccomp.json profile to be used as
546 the runtime's default. If not specified, then the internal default sec‐
547 comp profile will be used. (default: "")
548 --seccomp-use-default-when-empty: Use the default seccomp profile when
551 an empty one is specified
552 --selinux: Enable selinux support (default: false)
555 --separate-pull-cgroup="": [EXPERIMENTAL] Pull in new cgroup (default:
558 "")
559 --signature-policy="": Path to signature policy JSON file. (default:
562 "", to use the system-wide default)
563 --stats-collection-period="": The number of seconds between collecting
566 pod and container stats. If set to 0, the stats are collected on-demand
567 instead. (default: 0)
568 --storage-driver, -s="": OCI storage driver (default: "")
571 --storage-opt="": OCI storage driver option (default: [])
574 --stream-address="": Bind address for streaming socket (default:
577 127.0.0.1)
578 --stream-enable-tls: Enable encrypted TLS transport of the stream
581 server (default: false)
582 --stream-idle-timeout="": Length of time until open streams terminate
585 due to lack of activity
586 --stream-port="": Bind port for streaming socket. If the port is set to
589 '0', then CRI-O will allocate a random free port number. (default: 0)
590 --stream-tls-ca="": Path to the x509 CA(s) file used to verify and au‐
593 thenticate client communication with the encrypted stream. This file
594 can change and CRI-O will automatically pick up the changes within 5
595 minutes (default: "")
596 --stream-tls-cert="": Path to the x509 certificate file used to serve
599 the encrypted stream. This file can change and CRI-O will automatically
600 pick up the changes within 5 minutes (default: "")
601 --stream-tls-key="": Path to the key file used to serve the encrypted
604 stream. This file can change and CRI-O will automatically pick up the
605 changes within 5 minutes (default: "")
606 --tracing-endpoint="": Address on which the gRPC tracing collector will
609 listen (default: 0.0.0.0:4317)
610 --tracing-sampling-rate-per-million="": Number of samples to collect
613 per million OpenTelemetry spans (default: 0)
614 --uid-mappings="": Specify the UID mappings to use for the user name‐
617 space (default: "")
618 --version, -v: print the version
621 --version-file="": Location for CRI-O to lay down the temporary version
624 file. It is used to check if crio wipe should wipe containers, which
625 should always happen on a node reboot (default: /var/run/crio/version)
626 --version-file-persist="": Location for CRI-O to lay down the persis‐
629 tent version file. It is used to check if crio wipe should wipe images,
630 which should only happen when CRI-O has been upgraded (default:
631 /var/run/crio/version)
632
637 Generate bash, fish or zsh completions.
638
641 Generate the man page documentation.
642
645 Generate the markdown documentation.
646 --help, -h: show help
649
652 Outputs a commented version of the configuration file that could be
653 used by CRI-O. This allows you to save you current configuration setup
654 and then load it later with --config. Global options will modify the
655 output.
656 --default: Output the default configuration (without taking into ac‐
659 count any configuration options).
660 --migrate-defaults, -m="": Migrate the default config from a specified
663 version.
664 To run a config migration, just select the input config via the
665 global
666 '--config,-c' command line argument, for example:
667 crio -c /etc/crio/crio.conf.d/00-default.conf config -m 1.17
669 The migration will print converted configuration options to stderr
670 and will
671 output the resulting configuration to stdout.
672 Please note that the migration will overwrite any fields that have
673 changed
674 defaults between versions. To save a custom configuration change,
675 it should
676 be in a drop-in configuration file instead.
677 Possible values: "1.17" (default: 1.17)
678
681 display detailed version information
682 --json, -j: print JSON instead of text
685
688 wipe CRI-O's container and image storage
689 --force, -f: force wipe by skipping the version check
692
695 Shows a list of commands or help for one command
696
699 crio.conf (/etc/crio/crio.conf)
700 cri-o configuration file for all of the available command-line op‐
701 tions for
702 the crio(8) program, but in a TOML format that can be more easily
703 modified
704 and versioned.
705 policy.json (/etc/containers/policy.json)
708 Signature verification policy files are used to specify policy, e.g.
709 trusted
710 keys, applicable when deciding whether to accept an image, or indi‐
711 vidual
712 signatures of that image, as valid.
713 registries.conf (/etc/containers/registries.conf)
716 Registry configuration file specifies registries which are consulted
717 when
718 completing image names that do not include a registry or domain por‐
719 tion.
720 storage.conf (/etc/containers/storage.conf)
723 Storage configuration file specifies all of the available container
724 storage
725 options for tools using shared container storage.
726
730 crio.conf(5), crio.conf.d(5), oci-hooks(5), policy.json(5), reg‐
731 istries.conf(5), storage.conf(5)
732 crio(8)