1LORAX-COMPOSER(1) Lorax LORAX-COMPOSER(1)
2
6 lorax-composer - Lorax Composer Documentation
7 Authors
9 Brian C. Lane <bcl@redhat.com>
10 lorax-composer is an API server that allows you to build disk images
12 using Blueprints to describe the package versions to be installed into
13 the image. It is compatible with the Weldr project's bdcs-api REST
14 protocol. More information on Weldr can be found on the Weldr blog.
15 Behind the scenes it uses livemedia-creator and Anaconda to handle the
17 installation and configuration of the images.
18
20 · As of version 30.7 SELinux can be set to Enforcing. The current state
21 is logged for debugging purposes and if there are SELinux denials
22 they should be reported as a bug.
23 · All image types lock the root account, except for live-iso. You will
25 need to either use one of the Customizations methods for setting a
26 ssh key/password, install a package that creates a user, or use some‐
27 thing like cloud-init to setup access at boot time.
28
30 The best way to install lorax-composer is to use sudo dnf install
31 lorax-composer composer-cli, this will setup the weldr user and install
32 the systemd socket activation service. You will then need to enable it
33 with sudo systemctl enable lorax-composer.socket && sudo systemctl
34 start lorax-composer.socket. This will leave the server off until the
35 first request is made. Systemd will then launch the server and it will
36 remain running until the system is rebooted. This will cause some delay
37 in responding to the first request from the UI or composer-cli.
38 NOTE:
40 If you want lorax-composer to respond immediately to the first
41 request you can start and enable lorax-composer.service instead of
42 lorax-composer.socket
43
45 1. Create a weldr user and group by running useradd weldr
46 2. Remove any pre-existing socket directory with rm -rf /run/weldr/ A
48 new directory with correct permissions will be created the first
49 time the server runs.
50 3. Enable the socket activation with systemctl enable lorax-com‐
52 poser.socket && sudo systemctl start lorax-composer.socket.
53 NOTE: You can also run it directly with lorax-composer /path/to/blue‐
55 prints. However, lorax-composer does not react well to being started
56 both on the command line and via socket activation at the same time.
57 It is therefore recommended that you run it directly on the command
58 line only for testing or development purposes. For real use or devel‐
59 opment of other projects that simply use the API, you should stick to
60 socket activation only.
61 The /path/to/blueprints/ directory is where the blueprints' git repo
63 will be created, and all the blueprints created with the /api/v0/blue‐
64 prints/new route will be stored. If there are blueprint .toml files in
65 the top level of the directory they will be imported into the blueprint
66 git storage when lorax-composer starts.
67
69 Logs are stored under /var/log/lorax-composer/ and include all console
70 messages as well as extra debugging info and API requests.
71
73 Some security related issues that you should be aware of before running
74 lorax-composer:
75 · One of the API server threads needs to retain root privileges in
77 order to run Anaconda.
78 · Only allow authorized users access to the weldr group and socket.
80 Since Anaconda kickstarts are used there is the possibility that a user
82 could inject commands into a blueprint that would result in the kick‐
83 start executing arbitrary code on the host. Only authorized users
84 should be allowed to build images using lorax-composer.
85
87 Lorax Composer API Server
88 usage: lorax-composer [-h] [--socket SOCKET] [--user USER] [--group GROUP] [--log LOG] [--mockfiles MOCKFILES]
91 [--sharedir SHAREDIR] [-V] [-c CONFIG] [--releasever STRING] [--tmp TMP] [--proxy PROXY]
92 [--no-system-repos]
93 BLUEPRINTS
94 Positional Arguments
96 Path to the blueprints
97 Named Arguments
99 -socket
100 Path to the socket file to listen on
101 Default: "/run/weldr/api.socket"
103 -user User to use for reduced permissions
105 Default: "root"
107 -group Group to set ownership of the socket to
109 Default: "weldr"
111 -log Path to logfile (/var/log/lorax-composer/composer.log)
113 Default: "/var/log/lorax-composer/composer.log"
115 -mockfiles
117 Path to JSON files used for /api/mock/ paths
118 (/var/tmp/bdcs-mockfiles/)
119 Default: "/var/tmp/bdcs-mockfiles/"
121 -sharedir
123 Directory containing all the templates. Overrides config file
124 sharedir
125 V show program's version number and exit
127 Default: False
129 c, --config
131 Path to lorax-composer configuration file.
132 Default: "/etc/lorax/composer.conf"
134 -releasever
136 Release version to use for $releasever in dnf repository urls
137 -tmp Top level temporary directory
139 Default: "/var/tmp"
141 -proxy Set proxy for DNF, overrides configuration file setting.
143 -no-system-repos
145 Do not copy over system repos from /etc/yum.repos.d/ at startup
146 Default: False
148
150 The server runs as root, and as weldr. Communication with it is via a
151 unix domain socket (/run/weldr/api.socket by default). The directory
152 and socket are owned by root:weldr so that any user in the weldr group
153 can use the API to control lorax-composer.
154 At startup the server will check for the correct permissions and owner‐
156 ship of a pre-existing directory, or it will create a new one if it
157 doesn't exist. The socket path and group owner's name can be changed
158 from the cmdline by passing it the --socket and --group arguments.
159 It will then drop root privileges for the API thread and run as the
161 weldr user. The queue and compose thread still runs as root because it
162 needs to be able to mount/umount files and run Anaconda.
163
165 The welder-web GUI project can be used to construct blueprints and cre‐
166 ate composes using a web browser.
167 Or use the command line with composer-cli.
169
171 Blueprints are simple text files in TOML format that describe which
172 packages, and what versions, to install into the image. They can also
173 define a limited set of customizations to make to the final image.
174 Example blueprints can be found in the lorax-composer test suite, with
176 a simple one looking like this:
177 name = "base"
179 description = "A base system with bash"
180 version = "0.0.1"
181 [[packages]]
183 name = "bash"
184 version = "4.4.*"
185 The name field is the name of the blueprint. It can contain spaces, but
187 they will be converted to - when it is written to disk. It should be
188 short and descriptive.
189 description can be a longer description of the blueprint, it is only
191 used for display purposes.
192 version is a semver compatible version number. If a new blueprint is
194 uploaded with the same version the server will automatically bump the
195 PATCH level of the version. If the version doesn't match it will be
196 used as is. eg. Uploading a blueprint with version set to 0.1.0 when
197 the existing blueprint version is 0.0.1 will result in the new blue‐
198 print being stored as version 0.1.0.
199 [[packages]] and [[modules]]
201 These entries describe the package names and matching version glob to
202 be installed into the image.
203 The names must match the names exactly, and the versions can be an
205 exact match or a filesystem-like glob of the version using * wildcards
206 and ? character matching.
207 NOTE: Currently there are no differences between packages and modules
209 in lorax-composer. Both are treated like an rpm package dependency.
210 For example, to install tmux-2.9a and openssh-server-8.*, you would add
212 this to your blueprint:
213 [[packages]]
215 name = "tmux"
216 version = "2.9a"
217 [[packages]]
219 name = "openssh-server"
220 version = "8.*"
221 [[groups]]
223 The groups entries describe a group of packages to be installed into
224 the image. Package groups are defined in the repository metadata.
225 Each group has a descriptive name used primarily for display in user
226 interfaces and an ID more commonly used in kickstart files. Here, the
227 ID is the expected way of listing a group.
228 Groups have three different ways of categorizing their packages:
230 mandatory, default, and optional. For purposes of blueprints, manda‐
231 tory and default packages will be installed. There is no mechanism for
232 selecting optional packages.
233 For example, if you want to install the anaconda-tools group you would
235 add this to your blueprint:
236 [[groups]]
238 name="anaconda-tools"
239 groups is a TOML list, so each group needs to be listed separately,
241 like packages but with no version number.
242 Customizations
244 The [customizations] section can be used to configure the hostname of
245 the final image. eg.:
246 [customizations]
248 hostname = "baseimage"
249 This is optional and may be left out to use the defaults.
251 [customizations.kernel]
253 This allows you to append arguments to the bootloader's kernel command‐
254 line. This will not have any effect on tar or ext4-filesystem images
255 since they do not include a bootloader.
256 For example:
258 [customizations.kernel]
260 append = "nosmt=force"
261 [[customizations.sshkey]]
263 Set an existing user's ssh key in the final image:
264 [[customizations.sshkey]]
266 user = "root"
267 key = "PUBLIC SSH KEY"
268 The key will be added to the user's authorized_keys file.
270 WARNING:
272 key expects the entire content of ~/.ssh/id_rsa.pub
273 [[customizations.user]]
275 Add a user to the image, and/or set their ssh key. All fields for this
276 section are optional except for the name, here is a complete example:
277 [[customizations.user]]
279 name = "admin"
280 description = "Administrator account"
281 password = "$6$CHO2$3rN8eviE2t50lmVyBYihTgVRHcaecmeCk31L..."
282 key = "PUBLIC SSH KEY"
283 home = "/srv/widget/"
284 shell = "/usr/bin/bash"
285 groups = ["widget", "users", "wheel"]
286 uid = 1200
287 gid = 1200
288 If the password starts with $6$, $5$, or $2b$ it will be stored as an
290 encrypted password. Otherwise it will be treated as a plain text pass‐
291 word.
292 WARNING:
294 key expects the entire content of ~/.ssh/id_rsa.pub
295 [[customizations.group]]
297 Add a group to the image. name is required and gid is optional:
298 [[customizations.group]]
300 name = "widget"
301 gid = 1130
302 [customizations.timezone]
304 Customizing the timezone and the NTP servers to use for the system:
305 [customizations.timezone]
307 timezone = "US/Eastern"
308 ntpservers = ["0.north-america.pool.ntp.org", "1.north-america.pool.ntp.org"]
309 The values supported by timezone can be listed by running timedatectl
311 list-timezones.
312 If no timezone is setup the system will default to using UTC. The ntp
314 servers are also optional and will default to using the distribution
315 defaults which are fine for most uses.
316 In some image types there are already NTP servers setup, eg. Google
318 cloud image, and they cannot be overridden because they are required to
319 boot in the selected environment. But the timezone will be updated to
320 the one selected in the blueprint.
321 [customizations.locale]
323 Customize the locale settings for the system:
324 [customizations.locale]
326 languages = ["en_US.UTF-8"]
327 keyboard = "us"
328 The values supported by languages can be listed by running localectl
330 list-locales from the command line.
331 The values supported by keyboard can be listed by running localectl
333 list-keymaps from the command line.
334 Multiple languages can be added. The first one becomes the primary, and
336 the others are added as secondary. One or the other of languages or
337 keyboard must be included (or both) in the section.
338 [customizations.firewall]
340 By default the firewall blocks all access except for services that
341 enable their ports explicitly, like sshd. This command can be used to
342 open other ports or services. Ports are configured using the port:pro‐
343 tocol format:
344 [customizations.firewall]
346 ports = ["22:tcp", "80:tcp", "imap:tcp", "53:tcp", "53:udp"]
347 Numeric ports, or their names from /etc/services can be used in the
349 ports enabled/disabled lists.
350 The blueprint settings extend any existing settings in the image tem‐
352 plates, so if sshd is already enabled it will extend the list of ports
353 with the ones listed by the blueprint.
354 If the distribution uses firewalld you can specify services listed by
356 firewall-cmd --get-services in a customizations.firewall.services sec‐
357 tion:
358 [customizations.firewall.services]
360 enabled = ["ftp", "ntp", "dhcp"]
361 disabled = ["telnet"]
362 Remember that the firewall.services are different from the names in
364 /etc/services.
365 Both are optional, if they are not used leave them out or set them to
367 an empty list []. If you only want the default firewall setup this sec‐
368 tion can be omitted from the blueprint.
369 NOTE: The Google and OpenStack templates explicitly disable the fire‐
371 wall for their environment. This cannot be overridden by the blue‐
372 print.
373 [customizations.services]
375 This section can be used to control which services are enabled at boot
376 time. Some image types already have services enabled or disabled in
377 order for the image to work correctly, and cannot be overridden. eg.
378 ami requires sshd, chronyd, and cloud-init. Without them the image will
379 not boot. Blueprint services are added to, not replacing, the list
380 already in the templates, if any.
381 The service names are systemd service units. You may specify any sys‐
383 temd unit file accepted by systemctl enable eg. cockpit.socket:
384 [customizations.services]
386 enabled = ["sshd", "cockpit.socket", "httpd"]
387 disabled = ["postfix", "telnetd"]
388 [[repos.git]]
390 The [[repos.git]] entries are used to add files from a git repository
391 repository to the created image. The repository is cloned, the speci‐
392 fied ref is checked out and an rpm is created to install the files to a
393 destination path. The rpm includes a summary with the details of the
394 repository and reference used to create it. The rpm is also included in
395 the image build metadata.
396 To create an rpm named server-config-1.0-1.noarch.rpm you would add
398 this to your blueprint:
399 [[repos.git]]
401 rpmname="server-config"
402 rpmversion="1.0"
403 rpmrelease="1"
404 summary="Setup files for server deployment"
405 repo="PATH OF GIT REPO TO CLONE"
406 ref="v1.0"
407 destination="/opt/server/"
408 · rpmname: Name of the rpm to create, also used as the prefix name in
410 the tar archive
411 · rpmversion: Version of the rpm, eg. "1.0.0"
413 · rpmrelease: Release of the rpm, eg. "1"
415 · summary: Summary string for the rpm
417 · repo: URL of the get repo to clone and create the archive from
419 · ref: Git reference to check out. eg. origin/branch-name, git tag, or
421 git commit hash
422 · destination: Path to install the / of the git repo at when installing
424 the rpm
425 An rpm will be created with the contents of the git repository refer‐
427 enced, with the files being installed under /opt/server/ in this case.
428 ref can be any valid git reference for use with git archive. eg. to use
430 the head of a branch set it to origin/branch-name, a tag name, or a
431 commit hash.
432 Note that the repository is cloned in full each time a build is
434 started, so pointing to a repository with a large amount of history may
435 take a while to clone and use a significant amount of disk space. The
436 clone is temporary and is removed once the rpm is created.
437
439 livemedia-creator supports a large number of output types, and only
440 some of these are currently available via lorax-composer. To add a new
441 output type to lorax-composer a kickstart file needs to be added to
442 ./share/composer/. The name of the kickstart is what will be used by
443 the /compose/types route, and the compose_type field of the POST to
444 start a compose. It also needs to have code added to the pylo‐
445 rax.api.compose.compose_args() function. The _MAP entry in this func‐
446 tion defines what lorax-composer will pass to pylorax.install‐
447 er.novirt_install() when it runs the compose. When the compose is fin‐
448 ished the output files need to be copied out of the build directory
449 (/var/lib/lorax/composer/results/<UUID>/compose/), pylorax.api.com‐
450 pose.move_compose_results() handles this for each type. You should
451 move them instead of copying to save space.
452 If the new output type does not have support in livemedia-creator it
454 should be added there first. This will make the output available to the
455 widest number of users.
456 Example: Add partitioned disk support
458 Partitioned disk support is something that livemedia-creator already
459 supports via the --make-disk cmdline argument. To add this to
460 lorax-composer it needs 3 things:
461 · A partitioned-disk.ks file in ./share/composer/
463 · A new entry in the _MAP in pylorax.api.compose.compose_args()
465 · Add a bit of code to pylorax.api.compose.move_compose_results() to
467 move the disk image from the compose directory to the results direc‐
468 tory.
469 The partitioned-disk.ks is pretty similar to the example minimal kick‐
471 start in ./docs/fedora-minimal.ks. You should remove the url and repo
472 commands, they will be added by the compose process. Make sure the
473 bootloader packages are included in the %packages section at the end of
474 the kickstart, and you will want to leave off the %end so that the com‐
475 pose can append the list of packages from the blueprint.
476 The new _MAP entry should be a copy of one of the existing entries, but
478 with make_disk set to True. Make sure that none of the other make_*
479 options are True. The image_name is what the name of the final image
480 will be.
481 move_compose_results() can be as simple as moving the output file into
483 the results directory, or it could do some post-processing on it. The
484 end of the function should always clean up the ./compose/ directory,
485 removing any unneeded extra files. This is especially true for the
486 live-iso since it produces the contents of the iso as well as the
487 boot.iso itself.
488
490 By default lorax-composer uses the host's configured repositories. It
491 copies the *.repo files from /etc/yum.repos.d/ into /var/lib/lorax/com‐
492 poser/repos.d/ at startup, these are immutable system repositories and
493 cannot be deleted or changed. If you want to add additional repos you
494 can put them into /var/lib/lorax/composer/repos.d/ or use the
495 /api/v0/projects/source/* API routes to create them.
496 The new source can be added by doing a POST to the
498 /api/v0/projects/source/new route using JSON (with Content-Type header
499 set to application/json) or TOML (with it set to text/x-toml). The
500 format of the source looks like this (in TOML):
501 name = "custom-source-1"
503 url = "https://url/path/to/repository/"
504 type = "yum-baseurl"
505 proxy = "https://proxy-url/"
506 check_ssl = true
507 check_gpg = true
508 gpgkey_urls = ["https://url/path/to/gpg-key"]
509 The proxy and gpgkey_urls entries are optional. All of the others are
511 required. The supported types for the urls are:
512 · yum-baseurl is a URL to a yum repository.
514 · yum-mirrorlist is a URL for a mirrorlist.
516 · yum-metalink is a URL for a metalink.
518 If check_ssl is true the https certificates must be valid. If they are
520 self-signed you can either set this to false, or add your Certificate
521 Authority to the host system.
522 If check_gpg is true the GPG key must either be installed on the host
524 system, or gpgkey_urls should point to it.
525 You can edit an existing source (other than system sources), by doing a
527 POST to the new route with the new version of the source. It will over‐
528 write the previous one.
529 A list of existing sources is available from
531 /api/v0/projects/source/list, and detailed info on a source can be
532 retrieved with the /api/v0/projects/source/info/<source-name> route. By
533 default it returns JSON but it can also return TOML if ?format=toml is
534 added to the request.
535 Non-system sources can be deleted by doing a DELETE request to the
537 /api/v0/projects/source/delete/<source-name> route.
538 The documentation for the source API routes can be found here
540 The configured sources are used for all blueprint depsolve operations,
542 and for composing images. When adding additional sources you must make
543 sure that the packages in the source do not conflict with any other
544 package sources, otherwise depsolving will fail.
545 DVD ISO Package Source
547 In some situations the system may want to only use a DVD iso as the
548 package source, not the repos from the network. lorax-composer and ana‐
549 conda understand file:// URLs so you can mount an iso on the host, and
550 replace the system repo files with a configuration file pointing to the
551 DVD.
552 · Stop the lorax-composer.service if it is running
554 · Move the repo files in /etc/yum.repos.d/ someplace safe
556 · Create a new iso.repo file in /etc/yum.repos.d/:
558 [iso]
560 name=iso
561 baseurl=file:///mnt/iso/
562 enabled=1
563 gpgcheck=1
564 gpgkey=file:///mnt/iso/RPM-GPG-KEY-redhat-release
565 · Remove all the cached repo files from /var/lib/lorax/composer/repos/
567 · Restart the lorax-composer.service
569 · Check the output of composer-cli status show for any output specific
571 depsolve errors. For example, the DVD usually does not include
572 grub2-efi-*-cdboot-* so the live-iso image type will not be avail‐
573 able.
574 If you want to add the DVD source to the existing sources you can do
576 that by mounting the iso and creating a source file to point to it as
577 described in the Package Sources documentation. In that case there is
578 no need to remove the other sources from /etc/yum.repos.d/ or clear the
579 cached repos.
580
582 Weldr Team
583
585 2018, Red Hat, Inc.
58632.6 Feb 12, 2020 LORAX-COMPOSER(1)