1COMPOSER-CLI(1) Lorax COMPOSER-CLI(1)
2
6 composer-cli - Composer Cmdline Utility Documentation
7 Authors
9 Brian C. Lane <bcl@redhat.com>
10 composer-cli is an interactive tool for use with a WELDR API server,
12 managing blueprints, exploring available packages, and building new im‐
13 ages. As of Fedora 34, osbuild-composer <https://osbuild.org> is the
14 recommended server.
15 It requires the server to be installed on the local system, and the
17 user running it needs to be a member of the weldr group.
18
20 Lorax Composer commandline tool
21 usage: composer-cli [-h] [-j] [-s SOCKET] [--log LOG] [-a APIVER] [--test TESTMODE] [-V] ...
24 Positional Arguments
26 Named Arguments
27 j, --json
29 Output the raw JSON response instead of the normal output.
30 Default: False
32 s, --socket
34 Path to the socket file to listen on
35 Default: "/run/weldr/api.socket"
37 -log Path to logfile (./composer-cli.log)
39 a, --api
41 API Version to use
42 Default: "1"
44 -test Pass test mode to compose. 1=Mock compose with fail. 2=Mock com‐
46 pose with finished.
47 Default: 0
49 V show program's version number and exit
51 Default: False
53 compose start [--size XXXX] <BLUEPRINT> <TYPE> [<IMAGE-NAME> <PROVIDER>
56 <PROFILE> | <IMAGE-NAME> <PROFILE.TOML>]
57 Start a compose using the selected blueprint and output type.
58 Optionally start an upload. --size is supported by osbuild-com‐
59 poser, and is in MiB.
60 compose start-ostree [--size XXXX] [--parent PARENT] [--ref REF] <BLUE‐
62 PRINT> <TYPE> [<IMAGE-NAME> <PROFILE.TOML>]
63 Start an ostree compose using the selected blueprint and output
64 type. Optionally start an upload. This command is only supported
65 by osbuild-composer. --size is in MiB.
66 compose types
68 List the supported output types.
69 compose status
71 List the status of all running and finished composes.
72 compose list [waiting|running|finished|failed]
74 List basic information about composes.
75 compose log <UUID> [<SIZE>]
77 Show the last SIZE kB of the compose log.
78 compose cancel <UUID>
80 Cancel a running compose and delete any intermediate results.
81 compose delete <UUID,...>
83 Delete the listed compose results.
84 compose info <UUID>
86 Show detailed information on the compose.
87 compose metadata <UUID>
89 Download the metadata use to create the compose to <uuid>-meta‐
90 data.tar
91 compose logs <UUID>
93 Download the compose logs to <uuid>-logs.tar
94 compose results <UUID>
96 Download all of the compose results; metadata, logs, and image
97 to <uuid>.tar
98 compose image <UUID>
100 Download the output image from the compose. Filename depends on
101 the type.
102 blueprints list
104 List the names of the available blueprints.
105 blueprints show <BLUEPRINT,...>
107 Display the blueprint in TOML format.
108 blueprints changes <BLUEPRINT,...>
110 Display the changes for each blueprint.
111 blueprints diff <BLUEPRINT> <FROM-COMMIT> <TO-COMMIT>
113 Display the differences between 2 versions of a blueprint.
114 FROM-COMMIT can be a commit hash or NEWEST TO-COMMIT can be a
115 commit hash, NEWEST, or WORKSPACE
116 blueprints save <BLUEPRINT,...>
118 Save the blueprint to a file, <BLUEPRINT>.toml
119 blueprints delete <BLUEPRINT>
121 Delete a blueprint from the server
122 blueprints depsolve <BLUEPRINT,...>
124 Display the packages needed to install the blueprint.
125 blueprints push <BLUEPRINT>
127 Push a blueprint TOML file to the server.
128 blueprints freeze <BLUEPRINT,...>
130 Display the frozen blueprint's modules and packages.
131 blueprints freeze show <BLUEPRINT,...>
133 Display the frozen blueprint in TOML format.
134 blueprints freeze save <BLUEPRINT,...>
136 Save the frozen blueprint to a file, <blue‐
137 print-name>.frozen.toml.
138 blueprints tag <BLUEPRINT>
140 Tag the most recent blueprint commit as a release.
141 blueprints undo <BLUEPRINT> <COMMIT>
143 Undo changes to a blueprint by reverting to the selected commit.
144 blueprints workspace <BLUEPRINT>
146 Push the blueprint TOML to the temporary workspace storage.
147 modules list
149 List the available modules.
150 projects list
152 List the available projects.
153 projects info <PROJECT,...>
155 Show details about the listed projects.
156 sources list
158 List the available sources
159 sources info <SOURCE-NAME,...>
161 Details about the source.
162 sources add <SOURCE.TOML>
164 Add a package source to the server.
165 sources change <SOURCE.TOML>
167 Change an existing source
168 sources delete <SOURCE-NAME>
170 Delete a package source.
171 status show Show API server status.
173 upload info <UPLOAD-UUID>
175 Details about an upload
176 upload start <BUILD-UUID> <IMAGE-NAME> [<PROVIDER> <PROFILE>|<PRO‐
178 FILE.TOML>]
179 Upload a build image to the selected provider.
180 upload log <UPLOAD-UUID>
182 Show the upload log
183 upload cancel <UPLOAD-UUID>
185 Cancel an upload with that is queued or in progress
186 upload delete <UPLOAD-UUID>
188 Delete the upload and remove it from the build
189 upload reset <UPLOAD-UUID>
191 Reset the upload so that it can be tried again
192 providers list <PROVIDER>
194 List the available providers, or list the <provider's> available
195 profiles
196 providers show <PROVIDER> <PROFILE>
198 show the details of a specific provider's profile
199 providers push <PROFILE.TOML>
201 Add a new profile, or overwrite an existing one
202 providers save <PROVIDER> <PROFILE>
204 Save the profile's details to a TOML file named <PROFILE>.toml
205 providers delete <PROVIDER> <PROFILE>
207 Delete a profile from a provider
208
211 Start out by listing the available blueprints using composer-cli blue‐
212 prints list, pick one and save it to the local directory by running
213 composer-cli blueprints save http-server.
214 Edit the file (it will be saved with a .toml extension) and change the
216 description, add a package or module to it. Send it back to the server
217 by running composer-cli blueprints push http-server.toml. You can ver‐
218 ify that it was saved by viewing the changelog - composer-cli blue‐
219 prints changes http-server.
220 See the Example Blueprint for an example.
222
224 Build a qcow2 disk image from this blueprint by running composer-cli
225 compose start http-server qcow2. It will print a UUID that you can use
226 to keep track of the build. You can also cancel the build if needed.
227 The available types of images is displayed by composer-cli compose
229 types. Currently this consists of: alibaba, ami, ext4-filesystem,
230 google, hyper-v, live-iso, openstack, partitioned-disk, qcow2, tar,
231 vhd, vmdk
232 You can optionally start an upload of the finished image, see Image Up‐
234 loads for more information.
235
237 Monitor it using composer-cli compose status, which will show the sta‐
238 tus of all the builds on the system. You can view the end of the ana‐
239 conda build logs once it is in the RUNNING state using composer-cli
240 compose log UUID where UUID is the UUID returned by the start command.
241 Once the build is in the FINISHED state you can download the image.
243
245 Downloading the final image is done with composer-cli compose image
246 UUID and it will save the qcow2 image as UUID-disk.qcow2 which you can
247 then use to boot a VM like this:
248 qemu-kvm --name test-image -m 1024 -hda ./UUID-disk.qcow2
250
252 composer-cli can upload the images to a number of services, including
253 AWS, OpenStack, and vSphere. The upload can be started when the build
254 is finished, by using composer-cli compose start ... or an existing im‐
255 age can be uploaded with composer-cli upload start .... In order to ac‐
256 cess the service you need to pass authentication details to com‐
257 poser-cli using a TOML file, or reference a previously saved profile.
258 NOTE:
260 With osbuild-composer you can only specify upload targets during the
261 compose process.
262
264 Providers are the services providers with Ansible playbook support un‐
265 der /usr/share/lorax/lifted/providers/, you will need to gather some
266 provider specific information in order to authenticate with it. You can
267 view the required fields using composer-cli providers template
268 <PROVIDER>, eg. for AWS you would run:
269 composer-cli upload template aws
271 The output looks like this:
273 provider = "aws"
275 [settings]
277 aws_access_key = "AWS Access Key"
278 aws_bucket = "AWS Bucket"
279 aws_region = "AWS Region"
280 aws_secret_key = "AWS Secret Key"
281 Save this into an aws-credentials.toml file and use it when running
283 start.
284 AWS
286 The access key and secret key can be created by going to the
287 IAM->Users->Security Credentials section and creating a new access key.
288 The secret key will only be shown when it is first created so make sure
289 to record it in a secure place. The region should be the region that
290 you want to use the AMI in, and the bucket can be an existing bucket,
291 or a new one, following the normal AWS bucket naming rules. It will be
292 created if it doesn't already exist.
293 When uploading the image it is first uploaded to the s3 bucket, and
295 then converted to an AMI. If the conversion is successful the s3 ob‐
296 ject will be deleted. If it fails, re-trying after correcting the prob‐
297 lem will re-use the object if you have not deleted it in the meantime,
298 speeding up the process.
299
301 Profiles store the authentication settings associated with a specific
302 provider. Providers can have multiple profiles, as long as their names
303 are unique. For example, you may have one profile for testing and an‐
304 other for production uploads.
305 Profiles are created by pushing the provider settings template to the
307 server using composer-cli providers push <PROFILE.TOML> where PRO‐
308 FILE.TOML is the same as the provider template, but with the addition
309 of a profile field. For example, an AWS profile named test-uploads
310 would look like this:
311 provider = "aws"
313 profile = "test-uploads"
314 [settings]
316 aws_access_key = "AWS Access Key"
317 aws_bucket = "AWS Bucket"
318 aws_region = "AWS Region"
319 aws_secret_key = "AWS Secret Key"
320 You can view the profile by using composer-cli providers aws test-up‐
322 loads.
323
325 If you have a profile named test-uploads:
326 composer-cli compose start example-http-server ami "http image" aws test-uploads
328 Or if you have the settings stored in a TOML file:
330 composer-cli compose start example-http-server ami "http image" aws-settings.toml
332 It will return the UUID of the image build, and the UUID of the upload.
334 Once the build has finished successfully it will start the upload
335 process, which you can monitor with composer-cli upload info <UP‐
336 LOAD-UUID>
337 You can also view the upload logs from the Ansible playbook with:
339 ``composer-cli upload log <UPLOAD-UUID>``
341 The type of the image must match the type supported by the provider.
343
345 You can upload previously built images, as long as they are in the FIN‐
346 ISHED state, using composer-cli upload start ...`. If you have a pro‐
347 file named test-uploads:
348 composer-cli upload start <UUID> "http-image" aws test-uploads
350 Or if you have the settings stored in a TOML file:
352 composer-cli upload start <UUID> "http-image" aws-settings.toml
354 This will output the UUID of the upload, which can then be used to mon‐
356 itor the status in the same way described above.
357
359 There are a couple of arguments that can be helpful when debugging
360 problems. These are only meant for debugging and should not be used to
361 script access to the API. If you need to do that you can communicate
362 with it directly in the language of your choice.
363 --json will return the server's response as a nicely formatted json
365 output instead of printing what the command would usually print.
366 --test=1 will cause a compose start to start creating an image, and
368 then end with a failed state.
369 --test=2 will cause a compose to start and then end with a finished
371 state, without actually composing anything.
372
374 Blueprints are simple text files in TOML format that describe which
375 packages, and what versions, to install into the image. They can also
376 define a limited set of customizations to make to the final image.
377 A basic blueprint looks like this:
379 name = "base"
381 description = "A base system with bash"
382 version = "0.0.1"
383 [[packages]]
385 name = "bash"
386 version = "4.4.*"
387 The name field is the name of the blueprint. It can contain spaces, but
389 they will be converted to - when it is written to disk. It should be
390 short and descriptive.
391 description can be a longer description of the blueprint, it is only
393 used for display purposes.
394 version is a semver compatible version number. If a new blueprint is
396 uploaded with the same version the server will automatically bump the
397 PATCH level of the version. If the version doesn't match it will be
398 used as is. eg. Uploading a blueprint with version set to 0.1.0 when
399 the existing blueprint version is 0.0.1 will result in the new blue‐
400 print being stored as version 0.1.0.
401 [[packages]] and [[modules]]
403 These entries describe the package names and matching version glob to
404 be installed into the image.
405 The names must match the names exactly, and the versions can be an ex‐
407 act match or a filesystem-like glob of the version using * wildcards
408 and ? character matching.
409 NOTE:
411 Currently there are no differences between packages and modules in
412 osbuild-composer. Both are treated like an rpm package dependency.
413 For example, to install tmux-2.9a and openssh-server-8.*, you would add
415 this to your blueprint:
416 [[packages]]
418 name = "tmux"
419 version = "2.9a"
420 [[packages]]
422 name = "openssh-server"
423 version = "8.*"
424 [[groups]]
426 The groups entries describe a group of packages to be installed into
427 the image. Package groups are defined in the repository metadata.
428 Each group has a descriptive name used primarily for display in user
429 interfaces and an ID more commonly used in kickstart files. Here, the
430 ID is the expected way of listing a group.
431 Groups have three different ways of categorizing their packages:
433 mandatory, default, and optional. For purposes of blueprints, manda‐
434 tory and default packages will be installed. There is no mechanism for
435 selecting optional packages.
436 For example, if you want to install the anaconda-tools group you would
438 add this to your blueprint:
439 [[groups]]
441 name="anaconda-tools"
442 groups is a TOML list, so each group needs to be listed separately,
444 like packages but with no version number.
445 Customizations
447 The [customizations] section can be used to configure the hostname of
448 the final image. eg.:
449 [customizations]
451 hostname = "baseimage"
452 This is optional and may be left out to use the defaults.
454 [customizations.kernel]
456 This allows you to append arguments to the bootloader's kernel command‐
457 line. This will not have any effect on tar or ext4-filesystem images
458 since they do not include a bootloader.
459 For example:
461 [customizations.kernel]
463 append = "nosmt=force"
464 [[customizations.sshkey]]
466 Set an existing user's ssh key in the final image:
467 [[customizations.sshkey]]
469 user = "root"
470 key = "PUBLIC SSH KEY"
471 The key will be added to the user's authorized_keys file.
473 WARNING:
475 key expects the entire content of ~/.ssh/id_rsa.pub
476 [[customizations.user]]
478 Add a user to the image, and/or set their ssh key. All fields for this
479 section are optional except for the name, here is a complete example:
480 [[customizations.user]]
482 name = "admin"
483 description = "Administrator account"
484 password = "$6$CHO2$3rN8eviE2t50lmVyBYihTgVRHcaecmeCk31L..."
485 key = "PUBLIC SSH KEY"
486 home = "/srv/widget/"
487 shell = "/usr/bin/bash"
488 groups = ["widget", "users", "wheel"]
489 uid = 1200
490 gid = 1200
491 If the password starts with $6$, $5$, or $2b$ it will be stored as an
493 encrypted password. Otherwise it will be treated as a plain text pass‐
494 word.
495 WARNING:
497 key expects the entire content of ~/.ssh/id_rsa.pub
498 [[customizations.group]]
500 Add a group to the image. name is required and gid is optional:
501 [[customizations.group]]
503 name = "widget"
504 gid = 1130
505 [customizations.timezone]
507 Customizing the timezone and the NTP servers to use for the system:
508 [customizations.timezone]
510 timezone = "US/Eastern"
511 ntpservers = ["0.north-america.pool.ntp.org", "1.north-america.pool.ntp.org"]
512 The values supported by timezone can be listed by running timedatectl
514 list-timezones.
515 If no timezone is setup the system will default to using UTC. The ntp
517 servers are also optional and will default to using the distribution
518 defaults which are fine for most uses.
519 In some image types there are already NTP servers setup, eg. Google
521 cloud image, and they cannot be overridden because they are required to
522 boot in the selected environment. But the timezone will be updated to
523 the one selected in the blueprint.
524 [customizations.locale]
526 Customize the locale settings for the system:
527 [customizations.locale]
529 languages = ["en_US.UTF-8"]
530 keyboard = "us"
531 The values supported by languages can be listed by running localectl
533 list-locales from the command line.
534 The values supported by keyboard can be listed by running localectl
536 list-keymaps from the command line.
537 Multiple languages can be added. The first one becomes the primary, and
539 the others are added as secondary. One or the other of languages or
540 keyboard must be included (or both) in the section.
541 [customizations.firewall]
543 By default the firewall blocks all access except for services that en‐
544 able their ports explicitly, like sshd. This command can be used to
545 open other ports or services. Ports are configured using the port:pro‐
546 tocol format:
547 [customizations.firewall]
549 ports = ["22:tcp", "80:tcp", "imap:tcp", "53:tcp", "53:udp"]
550 Numeric ports, or their names from /etc/services can be used in the
552 ports enabled/disabled lists.
553 The blueprint settings extend any existing settings in the image tem‐
555 plates, so if sshd is already enabled it will extend the list of ports
556 with the ones listed by the blueprint.
557 If the distribution uses firewalld you can specify services listed by
559 firewall-cmd --get-services in a customizations.firewall.services sec‐
560 tion:
561 [customizations.firewall.services]
563 enabled = ["ftp", "ntp", "dhcp"]
564 disabled = ["telnet"]
565 Remember that the firewall.services are different from the names in
567 /etc/services.
568 Both are optional, if they are not used leave them out or set them to
570 an empty list []. If you only want the default firewall setup this sec‐
571 tion can be omitted from the blueprint.
572 NOTE: The Google and OpenStack templates explicitly disable the fire‐
574 wall for their environment. This cannot be overridden by the blue‐
575 print.
576 [customizations.services]
578 This section can be used to control which services are enabled at boot
579 time. Some image types already have services enabled or disabled in
580 order for the image to work correctly, and cannot be overridden. eg.
581 ami requires sshd, chronyd, and cloud-init. Without them the image will
582 not boot. Blueprint services are added to, not replacing, the list al‐
583 ready in the templates, if any.
584 The service names are systemd service units. You may specify any sys‐
586 temd unit file accepted by systemctl enable eg. cockpit.socket:
587 [customizations.services]
589 enabled = ["sshd", "cockpit.socket", "httpd"]
590 disabled = ["postfix", "telnetd"]
591 [[repos.git]]
593 NOTE:
594 Currently osbuild-composer does not support repos.git
595 The [[repos.git]] entries are used to add files from a git repository
597 repository to the created image. The repository is cloned, the speci‐
598 fied ref is checked out and an rpm is created to install the files to a
599 destination path. The rpm includes a summary with the details of the
600 repository and reference used to create it. The rpm is also included in
601 the image build metadata.
602 To create an rpm named server-config-1.0-1.noarch.rpm you would add
604 this to your blueprint:
605 [[repos.git]]
607 rpmname="server-config"
608 rpmversion="1.0"
609 rpmrelease="1"
610 summary="Setup files for server deployment"
611 repo="PATH OF GIT REPO TO CLONE"
612 ref="v1.0"
613 destination="/opt/server/"
614 • rpmname: Name of the rpm to create, also used as the prefix name in
616 the tar archive
617 • rpmversion: Version of the rpm, eg. "1.0.0"
619 • rpmrelease: Release of the rpm, eg. "1"
621 • summary: Summary string for the rpm
623 • repo: URL of the get repo to clone and create the archive from
625 • ref: Git reference to check out. eg. origin/branch-name, git tag, or
627 git commit hash
628 • destination: Path to install the / of the git repo at when installing
630 the rpm
631 An rpm will be created with the contents of the git repository refer‐
633 enced, with the files being installed under /opt/server/ in this case.
634 ref can be any valid git reference for use with git archive. eg. to use
636 the head of a branch set it to origin/branch-name, a tag name, or a
637 commit hash.
638 Note that the repository is cloned in full each time a build is
640 started, so pointing to a repository with a large amount of history may
641 take a while to clone and use a significant amount of disk space. The
642 clone is temporary and is removed once the rpm is created.
643
645 This example blueprint will install the tmux, git, and vim-enhanced
646 packages. It will set the root ssh key, add the widget and admin users
647 as well as a students group:
648 name = "example-custom-base"
650 description = "A base system with customizations"
651 version = "0.0.1"
652 [[packages]]
654 name = "tmux"
655 version = "*"
656 [[packages]]
658 name = "git"
659 version = "*"
660 [[packages]]
662 name = "vim-enhanced"
663 version = "*"
664 [customizations]
666 hostname = "custombase"
667 [[customizations.sshkey]]
669 user = "root"
670 key = "A SSH KEY FOR ROOT"
671 [[customizations.user]]
673 name = "widget"
674 description = "Widget process user account"
675 home = "/srv/widget/"
676 shell = "/usr/bin/false"
677 groups = ["dialout", "users"]
678 [[customizations.user]]
680 name = "admin"
681 description = "Widget admin account"
682 password = "$6$CHO2$3rN8eviE2t50lmVyBYihTgVRHcaecmeCk31LeOUleVK/R/aeWVHVZDi26zAH.o0ywBKH9Tc0/wm7sW/q39uyd1"
683 home = "/srv/widget/"
684 shell = "/usr/bin/bash"
685 groups = ["widget", "users", "students"]
686 uid = 1200
687 [[customizations.user]]
689 name = "plain"
690 password = "simple plain password"
691 [[customizations.user]]
693 name = "bart"
694 key = "SSH KEY FOR BART"
695 groups = ["students"]
696 [[customizations.group]]
698 name = "widget"
699 [[customizations.group]]
701 name = "students"
702
704 Weldr Team
705
707 2018, Red Hat, Inc.
70834.3 Oct 01, 2020 COMPOSER-CLI(1)