1LORAX-COMPOSER(1) Lorax LORAX-COMPOSER(1)
2
6 lorax-composer - Lorax Composer Documentation
7 Authors
9 Brian C. Lane <bcl@redhat.com>
10 lorax-composer is an API server that allows you to build disk images
12 using Blueprints to describe the package versions to be installed into
13 the image. It is compatible with the Weldr project's bdcs-api REST
14 protocol. More information on Weldr can be found on the Weldr blog.
15 Behind the scenes it uses livemedia-creator and Anaconda to handle the
17 installation and configuration of the images.
18
20 · As of version 30.7 SELinux can be set to Enforcing. The current state
21 is logged for debugging purposes and if there are SELinux denials
22 they should be reported as a bug.
23 · All image types lock the root account, except for live-iso. You will
25 need to either use one of the Customizations methods for setting a
26 ssh key/password, install a package that creates a user, or use some‐
27 thing like cloud-init to setup access at boot time.
28
30 The best way to install lorax-composer is to use sudo dnf install
31 lorax-composer composer-cli, this will setup the weldr user and install
32 the systemd socket activation service. You will then need to enable it
33 with sudo systemctl enable lorax-composer.socket && sudo systemctl
34 start lorax-composer.socket. This will leave the server off until the
35 first request is made. Systemd will then launch the server and it will
36 remain running until the system is rebooted. This will cause some delay
37 in responding to the first request from the UI or composer-cli.
38 NOTE:
40 If you want lorax-composer to respond immediately to the first
41 request you can start and enable lorax-composer.service instead of
42 lorax-composer.socket
43
45 1. Create a weldr user and group by running useradd weldr
46 2. Remove any pre-existing socket directory with rm -rf /run/weldr/ A
48 new directory with correct permissions will be created the first
49 time the server runs.
50 3. Enable the socket activation with systemctl enable lorax-com‐
52 poser.socket && sudo systemctl start lorax-composer.socket.
53 NOTE: You can also run it directly with lorax-composer /path/to/blue‐
55 prints. However, lorax-composer does not react well to being started
56 both on the command line and via socket activation at the same time.
57 It is therefore recommended that you run it directly on the command
58 line only for testing or development purposes. For real use or devel‐
59 opment of other projects that simply use the API, you should stick to
60 socket activation only.
61 The /path/to/blueprints/ directory is where the blueprints' git repo
63 will be created, and all the blueprints created with the /api/v0/blue‐
64 prints/new route will be stored. If there are blueprint .toml files in
65 the top level of the directory they will be imported into the blueprint
66 git storage when lorax-composer starts.
67
69 Logs are stored under /var/log/lorax-composer/ and include all console
70 messages as well as extra debugging info and API requests.
71
73 Some security related issues that you should be aware of before running
74 lorax-composer:
75 · One of the API server threads needs to retain root privileges in
77 order to run Anaconda.
78 · Only allow authorized users access to the weldr group and socket.
80 Since Anaconda kickstarts are used there is the possibility that a user
82 could inject commands into a blueprint that would result in the kick‐
83 start executing arbitrary code on the host. Only authorized users
84 should be allowed to build images using lorax-composer.
85
87 Lorax Composer API Server
88 usage: lorax-composer [-h] [--socket SOCKET] [--user USER] [--group GROUP]
91 [--log LOG] [--mockfiles MOCKFILES]
92 [--sharedir SHAREDIR] [-V] [-c CONFIG]
93 [--releasever STRING] [--tmp TMP] [--proxy PROXY]
94 [--no-system-repos]
95 BLUEPRINTS
96 Positional Arguments
98 Path to the blueprints
99 Named Arguments
101 -socket
102 Path to the socket file to listen on
103 Default: "/run/weldr/api.socket"
105 -user User to use for reduced permissions
107 Default: "root"
109 -group Group to set ownership of the socket to
111 Default: "weldr"
113 -log Path to logfile (/var/log/lorax-composer/composer.log)
115 Default: "/var/log/lorax-composer/composer.log"
117 -mockfiles
119 Path to JSON files used for /api/mock/ paths
120 (/var/tmp/bdcs-mockfiles/)
121 Default: "/var/tmp/bdcs-mockfiles/"
123 -sharedir
125 Directory containing all the templates. Overrides config file
126 sharedir
127 V show program's version number and exit
129 Default: False
131 c, --config
133 Path to lorax-composer configuration file.
134 Default: "/etc/lorax/composer.conf"
136 -releasever
138 Release version to use for $releasever in dnf repository urls
139 -tmp Top level temporary directory
141 Default: "/var/tmp"
143 -proxy Set proxy for DNF, overrides configuration file setting.
145 -no-system-repos
147 Do not copy over system repos from /etc/yum.repos.d/ at startup
148 Default: False
150
152 The server runs as root, and as weldr. Communication with it is via a
153 unix domain socket (/run/weldr/api.socket by default). The directory
154 and socket are owned by root:weldr so that any user in the weldr group
155 can use the API to control lorax-composer.
156 At startup the server will check for the correct permissions and owner‐
158 ship of a pre-existing directory, or it will create a new one if it
159 doesn't exist. The socket path and group owner's name can be changed
160 from the cmdline by passing it the --socket and --group arguments.
161 It will then drop root privileges for the API thread and run as the
163 weldr user. The queue and compose thread still runs as root because it
164 needs to be able to mount/umount files and run Anaconda.
165
167 The welder-web GUI project can be used to construct blueprints and cre‐
168 ate composes using a web browser.
169 Or use the command line with composer-cli.
171
173 Blueprints are simple text files in TOML format that describe which
174 packages, and what versions, to install into the image. They can also
175 define a limited set of customizations to make to the final image.
176 Example blueprints can be found in the lorax-composer test suite, with
178 a simple one looking like this:
179 name = "base"
181 description = "A base system with bash"
182 version = "0.0.1"
183 [[packages]]
185 name = "bash"
186 version = "4.4.*"
187 The name field is the name of the blueprint. It can contain spaces, but
189 they will be converted to - when it is written to disk. It should be
190 short and descriptive.
191 description can be a longer description of the blueprint, it is only
193 used for display purposes.
194 version is a semver compatible version number. If a new blueprint is
196 uploaded with the same version the server will automatically bump the
197 PATCH level of the version. If the version doesn't match it will be
198 used as is. eg. Uploading a blueprint with version set to 0.1.0 when
199 the existing blueprint version is 0.0.1 will result in the new blue‐
200 print being stored as version 0.1.0.
201 [[packages]] and [[modules]]
203 These entries describe the package names and matching version glob to
204 be installed into the image.
205 The names must match the names exactly, and the versions can be an
207 exact match or a filesystem-like glob of the version using * wildcards
208 and ? character matching.
209 NOTE: Currently there are no differences between packages and modules
211 in lorax-composer. Both are treated like an rpm package dependency.
212 For example, to install tmux-2.9a and openssh-server-8.*, you would add
214 this to your blueprint:
215 [[packages]]
217 name = "tmux"
218 version = "2.9a"
219 [[packages]]
221 name = "openssh-server"
222 version = "8.*"
223 [[groups]]
225 The groups entries describe a group of packages to be installed into
226 the image. Package groups are defined in the repository metadata.
227 Each group has a descriptive name used primarily for display in user
228 interfaces and an ID more commonly used in kickstart files. Here, the
229 ID is the expected way of listing a group.
230 Groups have three different ways of categorizing their packages:
232 mandatory, default, and optional. For purposes of blueprints, manda‐
233 tory and default packages will be installed. There is no mechanism for
234 selecting optional packages.
235 For example, if you want to install the anaconda-tools group you would
237 add this to your blueprint:
238 [[groups]]
240 name="anaconda-tools"
241 groups is a TOML list, so each group needs to be listed separately,
243 like packages but with no version number.
244 Customizations
246 The [customizations] section can be used to configure the hostname of
247 the final image. eg.:
248 [customizations]
250 hostname = "baseimage"
251 This is optional and may be left out to use the defaults.
253 [customizations.kernel]
255 This allows you to append arguments to the bootloader's kernel command‐
256 line. This will not have any effect on tar or ext4-filesystem images
257 since they do not include a bootloader.
258 For example:
260 [customizations.kernel]
262 append = "nosmt=force"
263 [[customizations.sshkey]]
265 Set an existing user's ssh key in the final image:
266 [[customizations.sshkey]]
268 user = "root"
269 key = "PUBLIC SSH KEY"
270 The key will be added to the user's authorized_keys file.
272 WARNING:
274 key expects the entire content of ~/.ssh/id_rsa.pub
275 [[customizations.user]]
277 Add a user to the image, and/or set their ssh key. All fields for this
278 section are optional except for the name, here is a complete example:
279 [[customizations.user]]
281 name = "admin"
282 description = "Administrator account"
283 password = "$6$CHO2$3rN8eviE2t50lmVyBYihTgVRHcaecmeCk31L..."
284 key = "PUBLIC SSH KEY"
285 home = "/srv/widget/"
286 shell = "/usr/bin/bash"
287 groups = ["widget", "users", "wheel"]
288 uid = 1200
289 gid = 1200
290 If the password starts with $6$, $5$, or $2b$ it will be stored as an
292 encrypted password. Otherwise it will be treated as a plain text pass‐
293 word.
294 WARNING:
296 key expects the entire content of ~/.ssh/id_rsa.pub
297 [[customizations.group]]
299 Add a group to the image. name is required and gid is optional:
300 [[customizations.group]]
302 name = "widget"
303 gid = 1130
304 [customizations.timezone]
306 Customizing the timezone and the NTP servers to use for the system:
307 [customizations.timezone]
309 timezone = "US/Eastern"
310 ntpservers = ["0.north-america.pool.ntp.org", "1.north-america.pool.ntp.org"]
311 The values supported by timezone can be listed by running timedatectl
313 list-timezones.
314 If no timezone is setup the system will default to using UTC. The ntp
316 servers are also optional and will default to using the distribution
317 defaults which are fine for most uses.
318 In some image types there are already NTP servers setup, eg. Google
320 cloud image, and they cannot be overridden because they are required to
321 boot in the selected environment. But the timezone will be updated to
322 the one selected in the blueprint.
323 [customizations.locale]
325 Customize the locale settings for the system:
326 [customizations.locale]
328 languages = ["en_US.UTF-8"]
329 keyboard = "us"
330 The values supported by languages can be listed by running localectl
332 list-locales from the command line.
333 The values supported by keyboard can be listed by running localectl
335 list-keymaps from the command line.
336 Multiple languages can be added. The first one becomes the primary, and
338 the others are added as secondary. One or the other of languages or
339 keyboard must be included (or both) in the section.
340 [customizations.firewall]
342 By default the firewall blocks all access except for services that
343 enable their ports explicitly, like sshd. This command can be used to
344 open other ports or services. Ports are configured using the port:pro‐
345 tocol format:
346 [customizations.firewall]
348 ports = ["22:tcp", "80:tcp", "imap:tcp", "53:tcp", "53:udp"]
349 Numeric ports, or their names from /etc/services can be used in the
351 ports enabled/disabled lists.
352 The blueprint settings extend any existing settings in the image tem‐
354 plates, so if sshd is already enabled it will extend the list of ports
355 with the ones listed by the blueprint.
356 If the distribution uses firewalld you can specify services listed by
358 firewall-cmd --get-services in a customizations.firewall.services sec‐
359 tion:
360 [customizations.firewall.services]
362 enabled = ["ftp", "ntp", "dhcp"]
363 disabled = ["telnet"]
364 Remember that the firewall.services are different from the names in
366 /etc/services.
367 Both are optional, if they are not used leave them out or set them to
369 an empty list []. If you only want the default firewall setup this sec‐
370 tion can be omitted from the blueprint.
371 NOTE: The Google and OpenStack templates explicitly disable the fire‐
373 wall for their environment. This cannot be overridden by the blue‐
374 print.
375 [customizations.services]
377 This section can be used to control which services are enabled at boot
378 time. Some image types already have services enabled or disabled in
379 order for the image to work correctly, and cannot be overridden. eg.
380 ami requires sshd, chronyd, and cloud-init. Without them the image will
381 not boot. Blueprint services are added to, not replacing, the list
382 already in the templates, if any.
383 The service names are systemd service units. You may specify any sys‐
385 temd unit file accepted by systemctl enable eg. cockpit.socket:
386 [customizations.services]
388 enabled = ["sshd", "cockpit.socket", "httpd"]
389 disabled = ["postfix", "telnetd"]
390 [[repos.git]]
392 The [[repos.git]] entries are used to add files from a git repository
393 repository to the created image. The repository is cloned, the speci‐
394 fied ref is checked out and an rpm is created to install the files to a
395 destination path. The rpm includes a summary with the details of the
396 repository and reference used to create it. The rpm is also included in
397 the image build metadata.
398 To create an rpm named server-config-1.0-1.noarch.rpm you would add
400 this to your blueprint:
401 [[repos.git]]
403 rpmname="server-config"
404 rpmversion="1.0"
405 rpmrelease="1"
406 summary="Setup files for server deployment"
407 repo="PATH OF GIT REPO TO CLONE"
408 ref="v1.0"
409 destination="/opt/server/"
410 · rpmname: Name of the rpm to create, also used as the prefix name in
412 the tar archive
413 · rpmversion: Version of the rpm, eg. "1.0.0"
415 · rpmrelease: Release of the rpm, eg. "1"
417 · summary: Summary string for the rpm
419 · repo: URL of the get repo to clone and create the archive from
421 · ref: Git reference to check out. eg. origin/branch-name, git tag, or
423 git commit hash
424 · destination: Path to install the / of the git repo at when installing
426 the rpm
427 An rpm will be created with the contents of the git repository refer‐
429 enced, with the files being installed under /opt/server/ in this case.
430 ref can be any valid git reference for use with git archive. eg. to use
432 the head of a branch set it to origin/branch-name, a tag name, or a
433 commit hash.
434 Note that the repository is cloned in full each time a build is
436 started, so pointing to a repository with a large amount of history may
437 take a while to clone and use a significant amount of disk space. The
438 clone is temporary and is removed once the rpm is created.
439
441 livemedia-creator supports a large number of output types, and only
442 some of these are currently available via lorax-composer. To add a new
443 output type to lorax-composer a kickstart file needs to be added to
444 ./share/composer/. The name of the kickstart is what will be used by
445 the /compose/types route, and the compose_type field of the POST to
446 start a compose. It also needs to have code added to the pylo‐
447 rax.api.compose.compose_args() function. The _MAP entry in this func‐
448 tion defines what lorax-composer will pass to pylorax.install‐
449 er.novirt_install() when it runs the compose. When the compose is fin‐
450 ished the output files need to be copied out of the build directory
451 (/var/lib/lorax/composer/results/<UUID>/compose/), pylorax.api.com‐
452 pose.move_compose_results() handles this for each type. You should
453 move them instead of copying to save space.
454 If the new output type does not have support in livemedia-creator it
456 should be added there first. This will make the output available to the
457 widest number of users.
458 Example: Add partitioned disk support
460 Partitioned disk support is something that livemedia-creator already
461 supports via the --make-disk cmdline argument. To add this to
462 lorax-composer it needs 3 things:
463 · A partitioned-disk.ks file in ./share/composer/
465 · A new entry in the _MAP in pylorax.api.compose.compose_args()
467 · Add a bit of code to pylorax.api.compose.move_compose_results() to
469 move the disk image from the compose directory to the results direc‐
470 tory.
471 The partitioned-disk.ks is pretty similar to the example minimal kick‐
473 start in ./docs/fedora-minimal.ks. You should remove the url and repo
474 commands, they will be added by the compose process. Make sure the
475 bootloader packages are included in the %packages section at the end of
476 the kickstart, and you will want to leave off the %end so that the com‐
477 pose can append the list of packages from the blueprint.
478 The new _MAP entry should be a copy of one of the existing entries, but
480 with make_disk set to True. Make sure that none of the other make_*
481 options are True. The image_name is what the name of the final image
482 will be.
483 move_compose_results() can be as simple as moving the output file into
485 the results directory, or it could do some post-processing on it. The
486 end of the function should always clean up the ./compose/ directory,
487 removing any unneeded extra files. This is especially true for the
488 live-iso since it produces the contents of the iso as well as the
489 boot.iso itself.
490
492 By default lorax-composer uses the host's configured repositories. It
493 copies the *.repo files from /etc/yum.repos.d/ into /var/lib/lorax/com‐
494 poser/repos.d/ at startup, these are immutable system repositories and
495 cannot be deleted or changed. If you want to add additional repos you
496 can put them into /var/lib/lorax/composer/repos.d/ or use the
497 /api/v0/projects/source/* API routes to create them.
498 The new source can be added by doing a POST to the
500 /api/v0/projects/source/new route using JSON (with Content-Type header
501 set to application/json) or TOML (with it set to text/x-toml). The
502 format of the source looks like this (in TOML):
503 name = "custom-source-1"
505 url = "https://url/path/to/repository/"
506 type = "yum-baseurl"
507 proxy = "https://proxy-url/"
508 check_ssl = true
509 check_gpg = true
510 gpgkey_urls = ["https://url/path/to/gpg-key"]
511 The proxy and gpgkey_urls entries are optional. All of the others are
513 required. The supported types for the urls are:
514 · yum-baseurl is a URL to a yum repository.
516 · yum-mirrorlist is a URL for a mirrorlist.
518 · yum-metalink is a URL for a metalink.
520 If check_ssl is true the https certificates must be valid. If they are
522 self-signed you can either set this to false, or add your Certificate
523 Authority to the host system.
524 If check_gpg is true the GPG key must either be installed on the host
526 system, or gpgkey_urls should point to it.
527 You can edit an existing source (other than system sources), by doing a
529 POST to the new route with the new version of the source. It will over‐
530 write the previous one.
531 A list of existing sources is available from
533 /api/v0/projects/source/list, and detailed info on a source can be
534 retrieved with the /api/v0/projects/source/info/<source-name> route. By
535 default it returns JSON but it can also return TOML if ?format=toml is
536 added to the request.
537 Non-system sources can be deleted by doing a DELETE request to the
539 /api/v0/projects/source/delete/<source-name> route.
540 The documentation for the source API routes can be found here
542 The configured sources are used for all blueprint depsolve operations,
544 and for composing images. When adding additional sources you must make
545 sure that the packages in the source do not conflict with any other
546 package sources, otherwise depsolving will fail.
547 DVD ISO Package Source
549 In some situations the system may want to only use a DVD iso as the
550 package source, not the repos from the network. lorax-composer and ana‐
551 conda understand file:// URLs so you can mount an iso on the host, and
552 replace the system repo files with a configuration file pointing to the
553 DVD.
554 · Stop the lorax-composer.service if it is running
556 · Move the repo files in /etc/yum.repos.d/ someplace safe
558 · Create a new iso.repo file in /etc/yum.repos.d/:
560 [iso]
562 name=iso
563 baseurl=file:///mnt/iso/
564 enabled=1
565 gpgcheck=1
566 gpgkey=file:///mnt/iso/RPM-GPG-KEY-redhat-release
567 · Remove all the cached repo files from /var/lib/lorax/composer/repos/
569 · Restart the lorax-composer.service
571 · Check the output of composer-cli status show for any output specific
573 depsolve errors. For example, the DVD usually does not include
574 grub2-efi-*-cdboot-* so the live-iso image type will not be avail‐
575 able.
576 If you want to add the DVD source to the existing sources you can do
578 that by mounting the iso and creating a source file to point to it as
579 described in the Package Sources documentation. In that case there is
580 no need to remove the other sources from /etc/yum.repos.d/ or clear the
581 cached repos.
582
584 Weldr Team
585
587 2018, Red Hat, Inc.
58833.2 Apr 28, 2020 LORAX-COMPOSER(1)