1LORAX-COMPOSER(1)                    Lorax                   LORAX-COMPOSER(1)
2
3
4

NAME

6       lorax-composer - Lorax Composer Documentation
7
8       Authors
9              Brian C. Lane <bcl@redhat.com>
10
11       lorax-composer  is  an  API server that allows you to build disk images
12       using Blueprints to describe the package versions to be installed  into
13       the  image.   It  is  compatible with the Weldr project's bdcs-api REST
14       protocol. More information on Weldr can be found on the Weldr blog.
15
16       Behind the scenes it uses livemedia-creator and Anaconda to handle  the
17       installation and configuration of the images.
18

IMPORTANT THINGS TO NOTE

20       · As of version 30.7 SELinux can be set to Enforcing. The current state
21         is logged for debugging purposes and if  there  are  SELinux  denials
22         they should be reported as a bug.
23
24       · All  image types lock the root account, except for live-iso. You will
25         need to either use one of the Customizations methods  for  setting  a
26         ssh key/password, install a package that creates a user, or use some‐
27         thing like cloud-init to setup access at boot time.
28

INSTALLATION

30       The best way to install lorax-composer  is  to  use  sudo  dnf  install
31       lorax-composer composer-cli, this will setup the weldr user and install
32       the systemd socket activation service. You will then need to enable  it
33       with  sudo  systemctl  enable  lorax-composer.socket  && sudo systemctl
34       start lorax-composer.socket. This will leave the server off  until  the
35       first  request is made. Systemd will then launch the server and it will
36       remain running until the system is rebooted. This will cause some delay
37       in responding to the first request from the UI or composer-cli.
38
39       NOTE:
40          If  you  want  lorax-composer  to  respond  immediately to the first
41          request you can start and enable lorax-composer.service  instead  of
42          lorax-composer.socket
43

QUICKSTART

45       1. Create a weldr user and group by running useradd weldr
46
47       2. Remove  any  pre-existing socket directory with rm -rf /run/weldr/ A
48          new directory with correct permissions will  be  created  the  first
49          time the server runs.
50
51       3. Enable  the  socket  activation  with  systemctl  enable  lorax-com‐
52          poser.socket && sudo systemctl start lorax-composer.socket.
53
54       NOTE: You can also run it directly with  lorax-composer  /path/to/blue‐
55       prints.   However,  lorax-composer does not react well to being started
56       both on the command line and via socket activation at  the  same  time.
57       It  is  therefore  recommended  that you run it directly on the command
58       line only for testing or development purposes.  For real use or  devel‐
59       opment  of  other projects that simply use the API, you should stick to
60       socket activation only.
61
62       The /path/to/blueprints/ directory is where the  blueprints'  git  repo
63       will  be created, and all the blueprints created with the /api/v0/blue‐
64       prints/new route will be stored.  If there are blueprint .toml files in
65       the top level of the directory they will be imported into the blueprint
66       git storage when lorax-composer starts.
67

LOGS

69       Logs are stored under /var/log/lorax-composer/ and include all  console
70       messages as well as extra debugging info and API requests.
71

SECURITY

73       Some security related issues that you should be aware of before running
74       lorax-composer:
75
76       · One of the API server threads needs  to  retain  root  privileges  in
77         order to run Anaconda.
78
79       · Only allow authorized users access to the weldr group and socket.
80
81       Since Anaconda kickstarts are used there is the possibility that a user
82       could inject commands into a blueprint that would result in  the  kick‐
83       start  executing  arbitrary  code  on  the host.  Only authorized users
84       should be allowed to build images using lorax-composer.
85

LORAX-COMPOSER CMDLINE ARGUMENTS

87       Lorax Composer API Server
88
89
90          usage: lorax-composer [-h] [--socket SOCKET] [--user USER] [--group GROUP]
91                                [--log LOG] [--mockfiles MOCKFILES]
92                                [--sharedir SHAREDIR] [-V] [-c CONFIG]
93                                [--releasever STRING] [--tmp TMP] [--proxy PROXY]
94                                [--no-system-repos]
95                                BLUEPRINTS
96
97   Positional Arguments
98       Path to the blueprints
99
100   Named Arguments
101       -socket
102              Path to the socket file to listen on
103
104              Default: "/run/weldr/api.socket"
105
106       -user  User to use for reduced permissions
107
108              Default: "root"
109
110       -group Group to set ownership of the socket to
111
112              Default: "weldr"
113
114       -log   Path to logfile (/var/log/lorax-composer/composer.log)
115
116              Default: "/var/log/lorax-composer/composer.log"
117
118       -mockfiles
119              Path   to    JSON    files    used    for    /api/mock/    paths
120              (/var/tmp/bdcs-mockfiles/)
121
122              Default: "/var/tmp/bdcs-mockfiles/"
123
124       -sharedir
125              Directory  containing  all  the templates. Overrides config file
126              sharedir
127
128       V      show program's version number and exit
129
130              Default: False
131
132       c, --config
133              Path to lorax-composer configuration file.
134
135              Default: "/etc/lorax/composer.conf"
136
137       -releasever
138              Release version to use for $releasever in dnf repository urls
139
140       -tmp   Top level temporary directory
141
142              Default: "/var/tmp"
143
144       -proxy Set proxy for DNF, overrides configuration file setting.
145
146       -no-system-repos
147              Do not copy over system repos from /etc/yum.repos.d/ at startup
148
149              Default: False
150

HOW IT WORKS

152       The server runs as root, and as weldr. Communication with it is  via  a
153       unix  domain  socket  (/run/weldr/api.socket by default). The directory
154       and socket are owned by root:weldr so that any user in the weldr  group
155       can use the API to control lorax-composer.
156
157       At startup the server will check for the correct permissions and owner‐
158       ship of a pre-existing directory, or it will create a  new  one  if  it
159       doesn't  exist.   The socket path and group owner's name can be changed
160       from the cmdline by passing it the --socket and --group arguments.
161
162       It will then drop root privileges for the API thread  and  run  as  the
163       weldr  user. The queue and compose thread still runs as root because it
164       needs to be able to mount/umount files and run Anaconda.
165

COMPOSING IMAGES

167       The welder-web GUI project can be used to construct blueprints and cre‐
168       ate composes using a web browser.
169
170       Or use the command line with composer-cli.
171

BLUEPRINTS

173       Blueprints  are  simple  text  files in TOML format that describe which
174       packages, and what versions, to install into the image. They  can  also
175       define a limited set of customizations to make to the final image.
176
177       Example  blueprints can be found in the lorax-composer test suite, with
178       a simple one looking like this:
179
180          name = "base"
181          description = "A base system with bash"
182          version = "0.0.1"
183
184          [[packages]]
185          name = "bash"
186          version = "4.4.*"
187
188       The name field is the name of the blueprint. It can contain spaces, but
189       they  will  be  converted to - when it is written to disk. It should be
190       short and descriptive.
191
192       description can be a longer description of the blueprint,  it  is  only
193       used for display purposes.
194
195       version  is  a  semver compatible version number. If a new blueprint is
196       uploaded with the same version the server will automatically  bump  the
197       PATCH  level  of  the  version. If the version doesn't match it will be
198       used as is. eg. Uploading a blueprint with version set  to  0.1.0  when
199       the  existing  blueprint  version is 0.0.1 will result in the new blue‐
200       print being stored as version 0.1.0.
201
202   [[packages]] and [[modules]]
203       These entries describe the package names and matching version  glob  to
204       be installed into the image.
205
206       The  names  must  match  the  names exactly, and the versions can be an
207       exact match or a filesystem-like glob of the version using *  wildcards
208       and ?  character matching.
209
210       NOTE:  As  of  lorax-composer-29.2-1 the versions are not used for dep‐
211       solving, that is planned for a future release. And currently there  are
212       no differences between packages and modules in lorax-composer.
213
214   [[groups]]
215       These  entries  describe  a  group of packages to be installed into the
216       image.  Package groups are defined in the  repository  metadata.   Each
217       group  has a descriptive name used primarily for display in user inter‐
218       faces and an ID more commonly used in kickstart files.  Here, the ID is
219       the expected way of listing a group.
220
221       Groups  have  three  different  ways  of  categorizing  their packages:
222       mandatory, default, and optional.  For purposes of  blueprints,  manda‐
223       tory and default packages will be installed.  There is no mechanism for
224       selecting optional packages.
225
226   Customizations
227       The [customizations] section can be used to configure the  hostname  of
228       the final image. eg.:
229
230          [customizations]
231          hostname = "baseimage"
232
233       This is optional and may be left out to use the defaults.
234
235   [customizations.kernel]
236       This allows you to append arguments to the bootloader's kernel command‐
237       line. This will not have any effect on tar  or  ext4-filesystem  images
238       since they do not include a bootloader.
239
240       For example:
241
242          [customizations.kernel]
243          append = "nosmt=force"
244
245   [[customizations.sshkey]]
246       Set an existing user's ssh key in the final image:
247
248          [[customizations.sshkey]]
249          user = "root"
250          key = "PUBLIC SSH KEY"
251
252       The key will be added to the user's authorized_keys file.
253
254       WARNING:
255          key expects the entire content of ~/.ssh/id_rsa.pub
256
257   [[customizations.user]]
258       Add a user to the image, and/or set their ssh key.  All fields for this
259       section are optional except for the name, here is a complete example:
260
261          [[customizations.user]]
262          name = "admin"
263          description = "Administrator account"
264          password = "$6$CHO2$3rN8eviE2t50lmVyBYihTgVRHcaecmeCk31L..."
265          key = "PUBLIC SSH KEY"
266          home = "/srv/widget/"
267          shell = "/usr/bin/bash"
268          groups = ["widget", "users", "wheel"]
269          uid = 1200
270          gid = 1200
271
272       If the password starts with $6$, $5$, or $2b$ it will be stored  as  an
273       encrypted  password. Otherwise it will be treated as a plain text pass‐
274       word.
275
276       WARNING:
277          key expects the entire content of ~/.ssh/id_rsa.pub
278
279   [[customizations.group]]
280       Add a group to the image. name is required and gid is optional:
281
282          [[customizations.group]]
283          name = "widget"
284          gid = 1130
285
286   [customizations.timezone]
287       Customizing the timezone and the NTP servers to use for the system:
288
289          [customizations.timezone]
290          timezone = "US/Eastern"
291          ntpservers = ["0.north-america.pool.ntp.org", "1.north-america.pool.ntp.org"]
292
293       The values supported by timezone can be listed by  running  timedatectl
294       list-timezones.
295
296       If  no  timezone is setup the system will default to using UTC. The ntp
297       servers are also optional and will default to  using  the  distribution
298       defaults which are fine for most uses.
299
300       In  some  image  types  there are already NTP servers setup, eg. Google
301       cloud image, and they cannot be overridden because they are required to
302       boot  in  the selected environment. But the timezone will be updated to
303       the one selected in the blueprint.
304
305   [customizations.locale]
306       Customize the locale settings for the system:
307
308          [customizations.locale]
309          languages = ["en_US.UTF-8"]
310          keyboard = "us"
311
312       The values supported by languages can be listed  by  running  localectl
313       list-locales from the command line.
314
315       The  values  supported  by  keyboard can be listed by running localectl
316       list-keymaps from the command line.
317
318       Multiple languages can be added. The first one becomes the primary, and
319       the  others  are  added  as secondary. One or the other of languages or
320       keyboard must be included (or both) in the section.
321
322   [customizations.firewall]
323       By default the firewall blocks all  access  except  for  services  that
324       enable  their  ports explicitly, like sshd. This command can be used to
325       open other ports or services. Ports are configured using the  port:pro‐
326       tocol format:
327
328          [customizations.firewall]
329          ports = ["22:tcp", "80:tcp", "imap:tcp", "53:tcp", "53:udp"]
330
331       Numeric  ports,  or  their  names from /etc/services can be used in the
332       ports enabled/disabled lists.
333
334       The blueprint settings extend any existing settings in the  image  tem‐
335       plates,  so if sshd is already enabled it will extend the list of ports
336       with the ones listed by the blueprint.
337
338       If the distribution uses firewalld you can specify services  listed  by
339       firewall-cmd  --get-services in a customizations.firewall.services sec‐
340       tion:
341
342          [customizations.firewall.services]
343          enabled = ["ftp", "ntp", "dhcp"]
344          disabled = ["telnet"]
345
346       Remember that the firewall.services are different  from  the  names  in
347       /etc/services.
348
349       Both  are  optional, if they are not used leave them out or set them to
350       an empty list []. If you only want the default firewall setup this sec‐
351       tion can be omitted from the blueprint.
352
353       NOTE:  The  Google and OpenStack templates explicitly disable the fire‐
354       wall for their environment.  This cannot be  overridden  by  the  blue‐
355       print.
356
357   [customizations.services]
358       This  section can be used to control which services are enabled at boot
359       time.  Some image types already have services enabled  or  disabled  in
360       order  for  the  image to work correctly, and cannot be overridden. eg.
361       ami requires sshd, chronyd, and cloud-init. Without them the image will
362       not  boot.  Blueprint  services  are  added to, not replacing, the list
363       already in the templates, if any.
364
365       The service names are systemd service units. You may specify  any  sys‐
366       temd unit file accepted by systemctl enable eg. cockpit.socket:
367
368          [customizations.services]
369          enabled = ["sshd", "cockpit.socket", "httpd"]
370          disabled = ["postfix", "telnetd"]
371
372   [[repos.git]]
373       The  [[repos.git]]  entries are used to add files from a git repository
374       repository to the created image. The repository is cloned,  the  speci‐
375       fied ref is checked out and an rpm is created to install the files to a
376       destination path. The rpm includes a summary with the  details  of  the
377       repository and reference used to create it. The rpm is also included in
378       the image build metadata.
379
380       To create an rpm named  server-config-1.0-1.noarch.rpm  you  would  add
381       this to your blueprint:
382
383          [[repos.git]]
384          rpmname="server-config"
385          rpmversion="1.0"
386          rpmrelease="1"
387          summary="Setup files for server deployment"
388          repo="PATH OF GIT REPO TO CLONE"
389          ref="v1.0"
390          destination="/opt/server/"
391
392       · rpmname:  Name  of the rpm to create, also used as the prefix name in
393         the tar archive
394
395       · rpmversion: Version of the rpm, eg. "1.0.0"
396
397       · rpmrelease: Release of the rpm, eg. "1"
398
399       · summary: Summary string for the rpm
400
401       · repo: URL of the get repo to clone and create the archive from
402
403       · ref: Git reference to check out. eg. origin/branch-name, git tag,  or
404         git commit hash
405
406       · destination: Path to install the / of the git repo at when installing
407         the rpm
408
409       An rpm will be created with the contents of the git  repository  refer‐
410       enced, with the files being installed under /opt/server/ in this case.
411
412       ref can be any valid git reference for use with git archive. eg. to use
413       the head of a branch set it to origin/branch-name, a  tag  name,  or  a
414       commit hash.
415
416       Note  that  the  repository  is  cloned  in  full  each time a build is
417       started, so pointing to a repository with a large amount of history may
418       take  a  while to clone and use a significant amount of disk space. The
419       clone is temporary and is removed once the rpm is created.
420

ADDING OUTPUT TYPES

422       livemedia-creator supports a large number of  output  types,  and  only
423       some  of these are currently available via lorax-composer. To add a new
424       output type to lorax-composer a kickstart file needs  to  be  added  to
425       ./share/composer/.  The  name  of the kickstart is what will be used by
426       the /compose/types route, and the compose_type field  of  the  POST  to
427       start  a  compose.  It  also  needs  to  have  code  added to the pylo‐
428       rax.api.compose.compose_args() function. The _MAP entry in  this  func‐
429       tion   defines   what  lorax-composer  will  pass  to  pylorax.install‐
430       er.novirt_install() when it runs the compose.  When the compose is fin‐
431       ished  the  output  files  need to be copied out of the build directory
432       (/var/lib/lorax/composer/results/<UUID>/compose/),     pylorax.api.com‐
433       pose.move_compose_results()  handles  this  for  each type.  You should
434       move them instead of copying to save space.
435
436       If the new output type does not have support  in  livemedia-creator  it
437       should be added there first. This will make the output available to the
438       widest number of users.
439
440   Example: Add partitioned disk support
441       Partitioned disk support is something  that  livemedia-creator  already
442       supports   via  the  --make-disk  cmdline  argument.  To  add  this  to
443       lorax-composer it needs 3 things:
444
445       · A partitioned-disk.ks file in ./share/composer/
446
447       · A new entry in the _MAP in pylorax.api.compose.compose_args()
448
449       · Add a bit of code  to  pylorax.api.compose.move_compose_results()  to
450         move  the disk image from the compose directory to the results direc‐
451         tory.
452
453       The partitioned-disk.ks is pretty similar to the example minimal  kick‐
454       start  in  ./docs/fedora-minimal.ks. You should remove the url and repo
455       commands, they will be added by the  compose  process.  Make  sure  the
456       bootloader packages are included in the %packages section at the end of
457       the kickstart, and you will want to leave off the %end so that the com‐
458       pose can append the list of packages from the blueprint.
459
460       The new _MAP entry should be a copy of one of the existing entries, but
461       with make_disk set to True. Make sure that none  of  the  other  make_*
462       options  are  True.  The image_name is what the name of the final image
463       will be.
464
465       move_compose_results() can be as simple as moving the output file  into
466       the  results  directory, or it could do some post-processing on it. The
467       end of the function should always clean up  the  ./compose/  directory,
468       removing  any  unneeded  extra  files.  This is especially true for the
469       live-iso since it produces the contents of  the  iso  as  well  as  the
470       boot.iso itself.
471

PACKAGE SOURCES

473       By  default  lorax-composer uses the host's configured repositories. It
474       copies the *.repo files from /etc/yum.repos.d/ into /var/lib/lorax/com‐
475       poser/repos.d/  at startup, these are immutable system repositories and
476       cannot be deleted or changed. If you want to add additional  repos  you
477       can   put   them   into  /var/lib/lorax/composer/repos.d/  or  use  the
478       /api/v0/projects/source/* API routes to create them.
479
480       The  new   source   can   be   added   by   doing   a   POST   to   the
481       /api/v0/projects/source/new  route using JSON (with Content-Type header
482       set to application/json) or TOML (with it  set  to  text/x-toml).   The
483       format of the source looks like this (in TOML):
484
485          name = "custom-source-1"
486          url = "https://url/path/to/repository/"
487          type = "yum-baseurl"
488          proxy = "https://proxy-url/"
489          check_ssl = true
490          check_gpg = true
491          gpgkey_urls = ["https://url/path/to/gpg-key"]
492
493       The  proxy  and gpgkey_urls entries are optional. All of the others are
494       required. The supported types for the urls are:
495
496       · yum-baseurl is a URL to a yum repository.
497
498       · yum-mirrorlist is a URL for a mirrorlist.
499
500       · yum-metalink is a URL for a metalink.
501
502       If check_ssl is true the https certificates must be valid. If they  are
503       self-signed  you  can either set this to false, or add your Certificate
504       Authority to the host system.
505
506       If check_gpg is true the GPG key must either be installed on  the  host
507       system, or gpgkey_urls should point to it.
508
509       You can edit an existing source (other than system sources), by doing a
510       POST to the new route with the new version of the source. It will over‐
511       write the previous one.
512
513       A     list     of     existing     sources     is     available    from
514       /api/v0/projects/source/list, and detailed info  on  a  source  can  be
515       retrieved with the /api/v0/projects/source/info/<source-name> route. By
516       default it returns JSON but it can also return TOML if ?format=toml  is
517       added to the request.
518
519       Non-system  sources  can  be  deleted  by doing a DELETE request to the
520       /api/v0/projects/source/delete/<source-name> route.
521
522       The documentation for the source API routes can be found here
523
524       The configured sources are used for all blueprint depsolve  operations,
525       and for composing images.  When adding additional sources you must make
526       sure that the packages in the source do not  conflict  with  any  other
527       package sources, otherwise depsolving will fail.
528
529   DVD ISO Package Source
530       In  some  situations  the  system may want to only use a DVD iso as the
531       package source, not the repos from the network. lorax-composer and ana‐
532       conda  understand file:// URLs so you can mount an iso on the host, and
533       replace the system repo files with a configuration file pointing to the
534       DVD.
535
536       · Stop the lorax-composer.service if it is running
537
538       · Move the repo files in /etc/yum.repos.d/ someplace safe
539
540       · Create a new iso.repo file in /etc/yum.repos.d/:
541
542            [iso]
543            name=iso
544            baseurl=file:///mnt/iso/
545            enabled=1
546            gpgcheck=1
547            gpgkey=file:///mnt/iso/RPM-GPG-KEY-redhat-release
548
549       · Remove all the cached repo files from /var/lib/lorax/composer/repos/
550
551       · Restart the lorax-composer.service
552
553       · Check  the output of composer-cli status show for any output specific
554         depsolve errors.  For example,  the  DVD  usually  does  not  include
555         grub2-efi-*-cdboot-*  so  the  live-iso image type will not be avail‐
556         able.
557
558       If you want to add the DVD source to the existing sources  you  can  do
559       that  by  mounting the iso and creating a source file to point to it as
560       described in the Package Sources documentation.  In that case there  is
561       no need to remove the other sources from /etc/yum.repos.d/ or clear the
562       cached repos.
563

AUTHOR

565       Weldr Team
566
568       2018, Red Hat, Inc.
569
570
571
572
57331.9                             Jul 29, 2019                LORAX-COMPOSER(1)
Impressum