1RDOPKG(1) RDOPKG(1)
2
6 rdopkg - RDO packaging tool
7
9 rdopkg <action> <arg>...
10 rdopkg -c
12 rdopkg -h
14
16 rdopkg is a tool for automating RDO/RHOSP packaging tasks, such as
17 introducing patches, updating packages to new versions and submitting
18 packages to RDO.
19 Run rdopkg -h to get available actions.
21 persistence
23 rdopkg provides multi-step actions where each step is (should be)
24 idempotent so if something fails along the way or human interaction is
25 required, rdopkg drops to shell, lets you fix the problem, and then
26 continue by running rdopkg --continue (rdopkg -c).
27 The state is stored in a file named .rdopkg.json in the current
29 directory. The last stored action can be inspected (rdopkg status),
30 continued (rdopkg --continue) or aborted (rdopkg --abort). rdopkg will
31 refuse to perform a new multi-step action if a state file is present.
32
34 Important actions diagram
35 +---------------------+
38 / WHAT DO YOU NEED? /
39 +-----+----------+----+
40 |
41 |
42 |
43 build a new package
44 |
45 v
46 +---------------------+
47 / what is changing? /
48 +----+-----+-----+----+
49 | | |
50 +------+ | +--------+
51 | | |
52 just update introduce update to new
53 .spec patch(es) upstream version
54 | | |
55 | | |
56 v v v
57 +-----+ +-------+ +-------------+
58 | fix | | patch | | new-version |
59 +-----+ +-------+ +-------------+
60 ACTION: fix
62 Apply changes to the .spec file.
63 Flow:
65 • Bump Release, prepare a new %changelog entry header.
67 • Drop to shell, let user edit the .spec file.
69 • After running rdopkg, ensure description was added to %changelog
71 and commit changes in a new commit.
72 Example:
74 cd python-novaclient
76 rdopkg fix
77 vim python-novaclient.spec
78 rdopkg -c
79 More use cases can be found in rdopkg-feature-fix(7).
81 ACTION: patch
83 Introduce new patches to the package.
84 This action works with the patches branch (see AUTOMAGIC → patches
86 branch).
87 By default, rdopkg resets the local patches branch to the remote
89 patches branch. You can skip this with -l/--local-patches to directly
90 use the local patches branch as is.
91 Don’t forget to git fetch before running the action.
93 After running rdopkg patch, the new commit will contain the changes if
95 there are any.
96 You can use -C/--changelog option to select how rdopkg detects new/old
98 patches and changelog message:
99 • detect: detect new/old patches using commit hash/subject (default)
101 • count: count old and new patches (doesn’t work for simultaneous
103 removal & addition of patches and rewriting -patches history)
104 • plain: just use generic "- Update patches" message
106 Or you can use -B/--no-bump to skip Release bump and changelog
108 generation and only update patch files and their references in the
109 .spec file. This is useful when you only need to align distgit with the
110 patches branch.
111 Use --amend to amend previous git commit with the changes and
113 autogenerated commit message instead of creating a new one. This is
114 very convenient when modifying distgit commits.
115 Please see AUTOMAGIC for additional information about using magic
117 patches_base and patches_ignore comments in your .spec file to
118 influcence patches generation.
119 Flow:
121 • Unless -l/--local-patches was used, reset the local patches branch
123 to the remote patches branch.
124 • Update patch files from local patches branch using git
126 format-patch.
127 • Update .spec file with correct patch files references.
129 • Unless ‘-B/--no-bump` was used, update .spec file: bump Release,
131 create new %changelog entry with new patches’ titles depending on
132 -C/--changelog option.
133 • If a %global commit asdf1234 macro declaration is present, rewrite
135 it with the current sha1 of the patches branch. (This makes the
136 sha1 value available during your package’s build process. You can
137 use this to build your program so that "mycoolprogram --version"
138 could display the sha1 to users.)
139 • Create new commit (or amend previous one with -a/--amend) with the
141 changes using %changelog to generate commit message if available.
142 • Display the diff.
144 Example:
146 rdopkg patch
148 rdopkg patch -lBa
149 More use cases can be found in rdopkg-feature-patch(7).
151 ACTION: update-patches
153 An alias for:
154 rdopkg patch --local-patches --no-bump
156 in the spirit of the ancient update-patches.sh script.
158 See ACTION: PATCH above.
160 ACTION: new-version
162 Update package to new upstream version.
163 This action works with the patches branch (see AUTOMAGIC → patches
165 branch). After a successful rebase, rdopkg will offer to push the
166 rebased patches branch.
167 Required new-version argument is a new version to rebase on, presumably
169 a git version tag.
170 Don’t forget to git fetch --all before running the action.
172 You can use the -N/--new-sources or -n/--no-new-sources options to
174 control whether new-version will run fedpkg new-sources (rhpkg
175 new-sources on Red Hat downstream products). By default, rdopkg will
176 automatically enable this step in following scenarios:
177 • Fedora distgit detected from origin git remote
179 (pkgs.fedoraproject.org)
180 • RH distgit detected from git branch (rhos-*, rh-*, ceph-*,
182 rhscon-*)
183 Otherwise, fedpkg new-sources is disabled (-n).
185 After running rdopkg new-version, a new commit will contain the
187 changes.
188 To only update .spec without touching patches branch, -b/--bump-only
190 can be used. Along with -n/--no-new-sources this enables local-only
191 operations, much like rpmdev-bumpspec:
192 rdopkg -bn
194 To note particular bugs in the changelog, use the -B/--bug option.
196 rdopkg will append the supplied string to changelog in brackets. For
197 example:
198 rdopkg new-version --bug rhbz#1234,rhbz#5678
200 will result in following %changelog line:
202 Update to 1.1.1 (rhbz#1234,rhbz#5678)
204 and corresponding Resolves: lines in commit message.
206 Flow:
208 • Show changes between the previous version and the current one,
210 especially modifications to requirements.txt.
211 • Reset the local patches branch to the remote patches branch
213 • Rebase the local patches branch on $NEW_VERSION tag.
215 • Update .spec file: set Version, Release and patches_base to
217 appropriate values and create a new %changelog entry.
218 • Download source tarball.
220 • Run fedpkg new-sources (rhpkg new-sources).
222 • Update patches from the local patches branch.
224 • Display the diff.
226 Example:
228 cd python-novaclient
230 git fetch --all
231 rdopkg new-version 2.15.0
232 # rebase failed, manually fix using git
233 rdopkg -c
234 More use cases can be found in rdopkg-feature-new-version(7).
236 ACTION: lint
238 Run checks for errors in current distgit.
239 Available checks selectable with --lint-checks:
241 • sanity: internal rdopkg sanity checks on the .spec
243 • rpmlint: run rpmlint tool on the .spec
245 • all: run all available checks (default)
247 Available error levels selectable with --error-level affect the exit
249 code:
250 • E: exit with code 23 when linting error is found (default)
252 • W: exit with code 23 when linttng error or warning is found
254 • -: only print errors/warnings, always returns 0
256 Most of the time you probably want just:
258 rdopkg lint
260 Example of only running rpmlint with W error level:
262 rdopkg lint --lint-checks rpmlint --error-level W
264 ACTION: clone
266 Clone an RDO package distgit and setup remotes.
267 clone uses rdoinfo metadata to clone the specified RDO package distgit
269 and also setup relevant remotes to get you packaging quickly.
270 If your github username differs from your $USER, use -u/--review-user.
272 Example:
274 rdopkg clone -u github-user python-novaclient
276 cd python-novaclient
277 git remote -v
278 ACTION: query
280 Query RDO/distro repos for available package versions.
281 See rdopkg-adv-requirements(7) for complete example of query and other
283 requirements management actions.
284 This action uses repoquery to discover the latest package versions
286 available from RDO and other repos available on a supported
287 distibution.
288 See output of rdopkg info for supported releases and distros.
290 Query specific RELEASE/DIST:
292 rdopkg query kilo/el7 openstack-nova
294 Query all dists of a release and show what’s happening:
296 rdopkg query -v kilo openstack-nova
298 ACTION: reqquery
300 Query RDO/distro repos for versions defined in requirements.txt.
301 See rdopkg-adv-requirements(7) for a complete example of reqquery and
303 other requirements management actions.
304 This action essentially runs rdopkg query on every module/package
306 defined in requirements.txt and prints a colorful report to quickly
307 find unmet dependencies. It accepts the same RELEAESE/DIST filter as
308 rdopkg query.
309 Python module names listed in requirements.txt are mapped to package
311 names using the rdopkg.actionmods.pymod2pkg module.
312 Query requirements.txt from 2015.1 tag:
314 rdopkg reqquery -R 2015.1 kilo/el7
316 Query requirements.txt file:
318 rdopkg reqquery -r path/to/requirements.txt kilo/f21
320 Query .spec Requires (experimental):
322 rdopkg reqquery -s
324 Verbosely dump query results to a file and view them:
326 rdopkg reqquery -v -d
328 rdopkg reqquery -l
329 ACTION: reqcheck
331 Inspect requirements.txt vs .spec Requires.
332 See rdopkg-adv-requirements(7) for complete example of reqcheck and
334 other requirements management actions.
335 This action parses the current requirements.txt from git and checks
337 whether they’re met in the .spec file. A simple report is produced.
338 Python module names listed in requirements.txt are mapped to package
340 names using rdopkg.actionmods.pymod2pkg module.
341 Use --spec/-s option to output Requires: suitable for pasting into
343 .spec files. Version comparisons are hidden, whitespace is detected
344 from .spec.
345 Use --strict/-S option to ask rdopkg to return an exit status. By
347 default, 0 is returned.
348 Example:
350 rdopkg reqcheck
352 rdopkg reqcheck -s
353 Override file
355 There are instances when you need to ignore some python modules from
357 being reqcheck'ed, or you might need to change the module version that
358 is in the requirements.txt file in order to successfully pass the
359 reqcheck operation. Some examples of these are: - a python module which
360 has been added in the requirements file but not yet packaged in RDO.
361 This module should be ignored during the packaging time period. - a
362 python module which has been updated in the requirements file but not
363 yet packaged in RDO. This module version should be replaced with the
364 one that is specified in the .spec file. - an obsolete python module
365 which still lives in the requirement file, and not in .spec file, is
366 tagged as MISSING in reqcheck output. It’s a false positive. The module
367 should be removed upstream, in the meantime, we should ignore it. -
368 when a module RPM has several Provide for the same subpackage. As
369 pymod2pkg currently only suports one py3pkg name for the translation,
370 this can bring you to have a missing module message displayed, even
371 though the module is present with another name (e.g PyYAML module)
372 So, there are only two operations: 1. replacing a version of a python
374 module; 2. ignoring (i.e deleting) a python module;
375 The adding operation is not handled. The packages that are in .spec
377 file and not in requirements file are tagged as EXCESS during reqcheck,
378 which does not make the reqcheck operation fail. That’s why adding
379 python module in the requirements file before reqcheck is not relevant.
380 To ignore python module or overwrite version during reqcheck you need
382 to use --override/-O option, and provides a YAML file, see below the
383 format:
384 ---
386 packages:
387 all:
388 - name: "python-yaml"
389 - name: "python-pbr"
390 version: ""
391 openstack-murano:
392 - name: "python-alembic"
393 version: ">= 0.9.6"
394 - name: "python-pbr"
395 version: ">= 2.0.0"
396 The first level describes the package name we want to override (e.g
398 openstack-murano). The second level includes the configuration of
399 python modules we want to replace or ignore.
400 The keyword all is reserved in first level. The configuration (list of
402 python modules) associated to all is applied on all packages. This can
403 be interesting when a rule is spotted several times (e.g PyYAML), there
404 is a high chance that this rule might be applied everytime. You declare
405 it globally in all and there is no need anymore to explicitly specify
406 it in a package configuration. However, if a rule with the same python
407 module name is found in specific package (e.g openstack-murano) and all
408 configuration, then the rule in the specific package configuration is
409 picked up. Note: all configuration should be used carefully as the
410 associated rules are applied globally.
411 In this example, for the package named openstack-murano: - rdopkg will
413 replace alembic with alembic>=0.9.6 in the requirements file before
414 comparing to the alembic Requires version in the .spec file. - there is
415 no version attribute for python-yaml which is in all configuration, so
416 it will be ignored during the reqcheck. The behavior is the same when
417 version attribute has an empty value. - python-pbr with empty version
418 in all configuration means it is ignored during reqcheck. But, as it
419 exists a rule with the same python module name which lives in
420 openstack-murano, python-pbr>=2.0.0 is picked up during reqcheck.
421 Note: This option does not actually write to disk when replacing or
423 ignoring python modules in the requirements file. reqcheck loads this
424 file on memory stream, then the option --override replaces or ignores
425 the modules provided in the YAML file in the stream. Then, this stream
426 is compared against the .spec file.
427 Example:
429 rdopkg reqcheck --override override-file.yml
431 ACTION: reqdiff
433 Show pretty diff of requirements.txt.
434 See rdopkg-adv-requirements(7) for a complete example of reqdiff and
436 other requirements management actions.
437 Use this to see how requirements changed between versions.
439 See diff between current and latest upstream version (automagic):
441 rdopkg reqdiff
443 See diff between current and specified version:
445 rdopkg reqdiff 2015.1
447 See diff between two supplied versions:
449 rdopkg reqdiff 2015.1 2015.2
451 ACTION: kojibuild
453 Build the package in koji.
454 Flow:
456 • Run equivalent of fedpkg build using disgusting fedpkg python
458 module.
459 • Watch the build.
461 Example:
463 rdopkg kojibuild
465 ACTION: amend
467 Amend last git commit with current dist-git changes and (re)generate
468 the commit message from %changelog.
469 This simple action is equivalent to running
471 git commit -a --amend -m "$AUTOMAGIC_COMMIT_MESSAGE"
473 See AUTOMAGIC → commit message for more information about the generated
475 commit message.
476 ACTION: squash
478 Squash last git commit into previous one. The commit message of the
479 previous commit is used.
480 This simple action is a shortcut for
482 git reset --soft HEAD~
484 git commit --amend --no-edit
485 This is useful for squashing commits created by lower level actions
487 such as update-patches.
488 ACTION: get-sources
490 Download package source archive.
491 Currently, Source0 from .spec file is downloaded.
493 ACTION: info
495 Show information about RDO packaging.
496 Use this command to find out about:
498 • currently supported RDO OpenStack releases
500 • which distros are supported for each release
502 • what branch to build from
504 • what build system to build in
506 • supported packages
508 • various repositories tied to a package
510 • package maintainers
512 This command is a human interface to rdoinfo.
514 Releases/dists/branches overview:
516 rdopkg info
518 Detailed information about a package:
520 rdopkg info novaclient
522 Filter packages by maintainers:
524 rdopkg info maintainers:jruzicka
526 ACTION: info-tags-diff
528 Show rdoinfo tag changes.
529 rdopkg info-tags-diff RDOINFODIR
531 will show per-package new/changed tags in rdoinfo between HEAD~..HEAD.
533 For an existing or new package, a list of changed tags is returned
535 Example:
537 $ rdopkg info-tags-diff ~/.rdopkg/rdoinfo
539 openstack-changed ['newton-uc', 'newton']
540 openstack-new-pkg ['under-review']
541 This is an interface to rdopkg.actionmods.rdoinfo:tags_diff().
543 ACTION: attr-tags-diff
545 Show rdoinfo attribute changes.
546 rdopkg info-attr-diff RDOINFODIR ATTRNAME [INFO_FILE]
548 will show per-package new/changed attributes in rdoinfo between
550 HEAD~..HEAD.
551 For an existing or new package, a list of changes in the attribute is
553 returned
554 Example:
556 $ rdopkg info-attr-diff ~/.rdopkg/rdoinfo upstream
558 openstack-changed 'https://github.com/openstack-changed-repo'
559 openstack-new-pkg 'https://github.com/openstack-new-pkg'
560 If ATTRNAME is not used anywhere in the distroinfo repository, or there
562 are no changes in the attributes, the command will return "No attribute
563 changes detected." and exit with RC=0.
564 You can optionally pass INFO_FILE as a parameter to define the
566 top-level YAML file to use.
567 This is an interface to rdopkg.actionmods.rdoinfo:attr_diff().
569 ACTION: findpkg
571 Find and show the single best matching package in rdoinfo.
572 This command produces the same output as rdopkg info but
574 • smart search is performed on package name, project name and
576 upstream URL
577 • only a single matching package is shown
579 Use -s/--strict to disable magic substring search and only match whole
581 fields.
582 This command is a human interface to distroinfo.query.find_package()
584 function.
585 Examples of usage:
587 rdopkg findpkg nova
589 rdopkg findpkg -s openstack-nova
590 rdopkg findpkg git://git.openstack.org/openstack/nova
591 rdopkg findpkg openstack/nova
592 rdopkg findpkg novacli
593 ACTION: conf
595 Display rdopkg’s local configuration.
596 This command prints the default configuration that ships with rdopkg
598 out of the box. You can override the individual settings here by using
599 .py files in the configuration directories.
600 Store your per-user configuration in ~/.rdopkg/conf.d/*.py, or store
602 system-wide configuration in /etc/rdopkg.d/*.py.
603 ACTION: tag-patches
605 Tag the local -patches branch with the package’s Name-Version-Release.
606 Since the -patches branch can change over time, including rebases,
608 rewrites, etc, we need a mechanism to keep historical records of what
609 the -patches branch looked like over time. Tagging the -patches branch
610 for each new NVR will maintain Git references to each snapshot of the
611 particular patches that went into each build.
612 To look at the -patches branch for an old build, you can simply "git
614 checkout name-version-release" for that build and get an exact
615 representation of the Git tree for that build.
616 If a previous tag exists with this name, rdopkg will exit with an error
618 unless you use the --force option to overwrite the existing tag with
619 this name.
620 You can automatically push the new tag with the --push option. It’s a
622 good idea to create and push the tag after every successful build.
623
625 Instead of requiring project config files or endless lists of command
626 line arguments, rdopkg tries to guess all the necessary variables.
627 patches branch
629 update-patches is a core lower level action for updating the dist-git
630 .spec file with patches from associated patches branch. rdopkg tries
631 hard to detect the patches branch automagically, it’s usually
632 $BRANCH-patches for $BRANCH dist-git but one patches branch per
633 multiple dist-gits is also supported.
634 Best illustrated by example, the following are all valid patches
636 branches for rhos-5.0-rhel-7 dist-git and they’re searched in that
637 order:
638 • rhos-5.0-rhel-7-patches
640 • rhos-5.0-rhel-patches
642 • rhos-5.0-patches ←-- preferred for RHOSP
644 • rhos-patches
646 Use rdopkg pkgenv to check detected patches branch.
648 You can specify remote patches branch by -p/--patches-branch action
650 parameter for actions that use it, such as patch and new-version.
651 You may explicitly set the name of your patches remote and patches
653 branch in your git configuration using the
654 rdopkg.<branch>.patches-remote and rdopkg.<branch>.patches-branch
655 options. For example, if you are working on a dist-git branch named
656 rhel-7.4 and you want to use rhel-7-patches for your patches branch,
657 you would run:
658 git config rdopkg.rhel-7.4.patches-branch rhel-7-patches
660 patches base
662 rdopkg calculates the git tag on which you are applying patches from
663 the Version tag in your .spec file. If your .spec file contains a macro
664 named milestone, the value of this macro will be appended to the
665 version. That is, if your spec file has:
666 Version: 2014.2.3
668 Then rdopkg will use 2014.2.3 as the base. If instead your .spec file
670 has:
671 %global milestone rc2
673 Version: 2014.2.3
675 Then rdopkg will use 2014.2.3rc2 as the base.
677 In older versions of rdopkg, it was necessary to explicitly set the
679 patch base using a special patches_base comment in your spec file. This
680 is now optional behavior (the patches base is calculated
681 automatically), but you can use this if you need to override the
682 automatic behavior.
683 The most common use of patches_base is to specify number of patches on
685 top of patches base (which defaults to spec Version) to skip:
686 # patches_base=+2
688 You can set an arbitrary git revision as a patches base:
690 # patches_base=1.2.3+2
692 You shouldn’t need to modify this by hand (other than perhaps the
694 number of skipped patches) as rdopkg manages patches_base as needed.
695 patches_ignore
697 update-patches also supports filtering out patches based on matching a
698 regex provided by a magic #patches_ignore comment in the spec file.
699 This is useful, for example, in case the patches branch contains
700 changes that are related to the CI/code review infra, that are useful
701 to keep around but don’t need to end up in the RPM.
702 For example, if you add the following comments in your package’s .spec
704 file:
705 # patches_base=10.2.5
707 # patches_ignore=DROP-IN-RPM
708 then rdopkg will not create .patch files for any commits that have
710 "DROP-IN-RPM" in the Git commit log’s subject line.
711 Note: these lines should be directly above any Patch000X lines in your
713 .spec file.
714 Release bumping
716 rdopkg fix and rdopkg patch bump the Release tag in .spec file.
717 By default, last numeric only part of Release is bumped:
719 1.1.1.a.b.c -> 1.1.2.a.b.c
721 You can override this using -R/--release-bump-index argument which
723 expects MAJOR/MINOR/PATCH or integer index of release part to bump,
724 starting at 1 from the left:
725 -R 1 / -R MAJOR: 1.1.1 -> 2.1.1
727 -R 2 / -R MINOR: 1.1.1 -> 1.2.1
728 -R 3 / -R PATCH: 1.1.1 -> 1.1.2
729 -R 4: 1.1.1.1 -> 1.1.1.2
730 -R 5: 1.1.1.1.1 -> 1.1.1.1.2
731 ...
732 DLRN 0.date.hash and 0.1.date.hash formats are detected and default to
734 bumping 2nd Release part (-R 2).
735 commit message
737 Commit messages created by rdopkg are generated from .spec file Name,
738 Version and Release (NVR) as well as last %changelog entry.
739 All rdopkg actions that modify distgit use following format:
741 package-name-1.2.3-4
743 Changelog:
745 - Doom the World (rhbz#111111)
746 - Fix Impending Doom support
747 - Save the World (rhbz#222222)
748 Resolves: rhbz#111111
750 Resolves: rhbz#222222
751 Change-Id: deadbeedeadbeedeadbeedeadbeedeadbeedeadbee
752 For each (rhbz#XYZ) mentioned in latest %changelog entry, Resolves:
754 rhbz#XYZ line is appended to commit message as required by RHOSP
755 workflow.
756 protip: To (re)generate nice commit message after modifying .spec file,
758 use rdopkg amend (see ACTION: amend above).
759 For example, following %changelog entry:
761 %changelog
763 * Tue Feb 11 2014 Jakub Ruzicka <jruzicka@redhat.com> 0.5.0-1
764 - Update to upstream 0.5.0
765 - Fix evil Bug of Doom (rhbz#123456)
766 will generate following commit message:
768 package-name-0.5.0-1
770 Changelog:
772 - Update to upstream 0.5.0
773 - Fix evil Bug of Doom (rhbz#123456)
774 Resolves: rhbz#123456
776 rdoinfo
778 rdoinfo is a special utility repository with RDO metadata:
779 https://github.com/redhat-openstack/rdoinfo
781 rdopkg uses rdoinfo to
783 • detect release/dist from branch name
785 • check valid RDO updates
787 • query packages from RDO/distribution repos
789 and more.
791 You can view the rdoinfo metada using rdopkg info.
793
795 rdopkg-adv-new-version(7), rdopkg-adv-requirements(7),
796 rdopkg-feature-pkgenv(7), rdopkg-feature-fix(7),
797 rdopkg-feature-patch(7), rdopkg-feature-new-version(7),
798 rdopkg-feature-actions(7)
799
801 rdopkg is maintained by Jakub Ruzicka <jruzicka@redhat.com>.
802 Bugs are tracked as github Issues:
804 https://github.com/softwarefactory-project/rdopkg/issues
806 To report a new bug:
808 https://github.com/softwarefactory-project/rdopkg/issues/new
810 07/23/2021 RDOPKG(1)