1RDOPKG(1) RDOPKG(1)
2
6 rdopkg - RDO packaging tool
7
9 rdopkg <action> <arg>...
10 rdopkg -c
12 rdopkg -h
14
16 rdopkg is a tool for automating RDO/RHOSP packaging tasks, such as
17 introducing patches, updating packages to new versions and submitting
18 packages to RDO.
19 Run rdopkg -h to get available actions.
21 persistence
23 rdopkg provides multi-step actions where each step is (should be)
24 idempotent so if something fails along the way or human interaction is
25 required, rdopkg drops to shell, lets you fix the problem, and then
26 continue by running rdopkg --continue (rdopkg -c).
27 The state is stored in a file named .rdopkg.json in the current
29 directory. The last stored action can be inspected (rdopkg status),
30 continued (rdopkg --continue) or aborted (rdopkg --abort). rdopkg will
31 refuse to perform a new multi-step action if a state file is present.
32
34 Important actions diagram
35 +---------------------+
38 / WHAT DO YOU NEED? /
39 +-----+----------+----+
40 |
41 |
42 |
43 build a new package
44 |
45 v
46 +---------------------+
47 / what is changing? /
48 +----+-----+-----+----+
49 | | |
50 +------+ | +--------+
51 | | |
52 just update introduce update to new
53 .spec patch(es) upstream version
54 | | |
55 | | |
56 v v v
57 +-----+ +-------+ +-------------+
58 | fix | | patch | | new-version |
59 +-----+ +-------+ +-------------+
60 ACTION: fix
62 Apply changes to the .spec file.
63 Flow:
65 • Bump Release, prepare a new %changelog entry header.
67 • Drop to shell, let user edit the .spec file.
69 • After running rdopkg, ensure description was added to %changelog
71 and commit changes in a new commit.
72 Example:
74 cd python-novaclient
76 rdopkg fix
77 vim python-novaclient.spec
78 rdopkg -c
79 More use cases can be found in rdopkg-feature-fix(7).
81 ACTION: patch
83 Introduce new patches to the package.
84 This action works with the patches branch (see AUTOMAGIC → patches
86 branch).
87 By default, rdopkg resets the local patches branch to the remote
89 patches branch. You can skip this with -l/--local-patches to directly
90 use the local patches branch as is.
91 Don’t forget to git fetch before running the action.
93 After running rdopkg patch, the new commit will contain the changes if
95 there are any.
96 You can use -C/--changelog option to select how rdopkg detects new/old
98 patches and changelog message:
99 • detect: detect new/old patches using commit hash/subject (default)
101 • count: count old and new patches (doesn’t work for simultaneous
103 removal & addition of patches and rewriting -patches history)
104 • plain: just use generic "- Update patches" message
106 Or you can use -B/--no-bump to skip Release bump and changelog
108 generation and only update patch files and their references in the
109 .spec file. This is useful when you only need to align distgit with the
110 patches branch.
111 Use --amend to amend previous git commit with the changes and
113 autogenerated commit message instead of creating a new one. This is
114 very convenient when modifying distgit commits.
115 Please see AUTOMAGIC for additional information about using magic
117 patches_base and patches_ignore comments in your .spec file to
118 influcence patches generation.
119 Flow:
121 • Unless -l/--local-patches was used, reset the local patches branch
123 to the remote patches branch.
124 • Update patch files from local patches branch using git
126 format-patch.
127 • Update .spec file with correct patch files references.
129 • Unless ‘-B/--no-bump` was used, update .spec file: bump Release,
131 create new %changelog entry with new patches’ titles depending on
132 -C/--changelog option.
133 • If a %global commit asdf1234 macro declaration is present, rewrite
135 it with the current sha1 of the patches branch. (This makes the
136 sha1 value available during your package’s build process. You can
137 use this to build your program so that "mycoolprogram --version"
138 could display the sha1 to users.)
139 • Create new commit (or amend previous one with -a/--amend) with the
141 changes using %changelog to generate commit message if available.
142 • Display the diff.
144 Example:
146 rdopkg patch
148 rdopkg patch -lBa
149 More use cases can be found in rdopkg-feature-patch(7).
151 ACTION: update-patches
153 An alias for:
154 rdopkg patch --local-patches --no-bump
156 in the spirit of the ancient update-patches.sh script.
158 See ACTION: PATCH above.
160 ACTION: new-version
162 Update package to new upstream version.
163 This action works with the patches branch (see AUTOMAGIC → patches
165 branch). After a successful rebase, rdopkg will offer to push the
166 rebased patches branch.
167 Required new-version argument is a new version to rebase on, presumably
169 a git version tag.
170 Don’t forget to git fetch --all before running the action.
172 You can use the -N/--new-sources or -n/--no-new-sources options to
174 control whether new-version will run fedpkg new-sources (rhpkg
175 new-sources on Red Hat downstream products). By default, rdopkg will
176 automatically enable this step in following scenarios:
177 • Fedora distgit detected from origin git remote
179 (pkgs.fedoraproject.org)
180 • RH distgit detected from git branch (rhos-*, rh-*, ceph-*,
182 rhscon-*)
183 Otherwise, fedpkg new-sources is disabled (-n).
185 After running rdopkg new-version, a new commit will contain the
187 changes.
188 To only update .spec without touching patches branch, -b/--bump-only
190 can be used. Along with -n/--no-new-sources this enables local-only
191 operations, much like rpmdev-bumpspec:
192 rdopkg -bn
194 To note particular bugs in the changelog, use the -B/--bug option.
196 rdopkg will append the supplied string to changelog in brackets. For
197 example:
198 rdopkg new-version --bug rhbz#1234,rhbz#5678
200 will result in following %changelog line:
202 Update to 1.1.1 (rhbz#1234,rhbz#5678)
204 and corresponding Resolves: lines in commit message.
206 Flow:
208 • Show changes between the previous version and the current one,
210 especially modifications to requirements.txt.
211 • Reset the local patches branch to the remote patches branch
213 • Rebase the local patches branch on $NEW_VERSION tag.
215 • Update .spec file: set Version, Release and patches_base to
217 appropriate values and create a new %changelog entry.
218 • Download source tarball.
220 • Run fedpkg new-sources (rhpkg new-sources).
222 • Update patches from the local patches branch.
224 • Display the diff.
226 Example:
228 cd python-novaclient
230 git fetch --all
231 rdopkg new-version 2.15.0
232 # rebase failed, manually fix using git
233 rdopkg -c
234 More use cases can be found in rdopkg-feature-new-version(7).
236 ACTION: lint
238 Run checks for errors in current distgit.
239 Available checks selectable with --lint-checks:
241 • sanity: internal rdopkg sanity checks on the .spec
243 • rpmlint: run rpmlint tool on the .spec
245 • all: run all available checks (default)
247 Available error levels selectable with --error-level affect the exit
249 code:
250 • E: exit with code 23 when linting error is found (default)
252 • W: exit with code 23 when linttng error or warning is found
254 • -: only print errors/warnings, always returns 0
256 Most of the time you probably want just:
258 rdopkg lint
260 Example of only running rpmlint with W error level:
262 rdopkg lint --lint-checks rpmlint --error-level W
264 ACTION: clone
266 Clone an RDO package distgit and setup remotes.
267 clone uses rdoinfo metadata to clone the specified RDO package distgit
269 and also setup relevant remotes to get you packaging quickly.
270 If your github username differs from your $USER, use -u/--review-user.
272 Example:
274 rdopkg clone -u github-user python-novaclient
276 cd python-novaclient
277 git remote -v
278 ACTION: query
280 Query RDO/distro repos for available package versions.
281 See rdopkg-adv-requirements(7) for complete example of query and other
283 requirements management actions.
284 This action uses repoquery to discover the latest package versions
286 available from RDO and other repos available on a supported
287 distibution.
288 See output of rdopkg info for supported releases and distros.
290 Query specific RELEASE/DIST:
292 rdopkg query kilo/el7 openstack-nova
294 Query all dists of a release and show what’s happening:
296 rdopkg query -v kilo openstack-nova
298 ACTION: reqquery
300 Query RDO/distro repos for versions defined in requirements.txt.
301 See rdopkg-adv-requirements(7) for a complete example of reqquery and
303 other requirements management actions.
304 This action essentially runs rdopkg query on every module/package
306 defined in requirements.txt and prints a colorful report to quickly
307 find unmet dependencies. It accepts the same RELEAESE/DIST filter as
308 rdopkg query.
309 Python module names listed in requirements.txt are mapped to package
311 names using the rdopkg.actionmods.pymod2pkg module.
312 Query requirements.txt from 2015.1 tag:
314 rdopkg reqquery -R 2015.1 kilo/el7
316 Query requirements.txt file:
318 rdopkg reqquery -r path/to/requirements.txt kilo/f21
320 Query .spec Requires (experimental):
322 rdopkg reqquery -s
324 Verbosely dump query results to a file and view them:
326 rdopkg reqquery -v -d
328 rdopkg reqquery -l
329 ACTION: reqcheck
331 Inspect requirements.txt vs .spec Requires.
332 See rdopkg-adv-requirements(7) for complete example of reqcheck and
334 other requirements management actions.
335 This action parses the current requirements.txt from git and checks
337 whether they’re met in the .spec file. A simple report is produced.
338 Python module names listed in requirements.txt are mapped to package
340 names using rdopkg.actionmods.pymod2pkg module.
341 Use --spec/-s option to output Requires: suitable for pasting into
343 .spec files. Version comparisons are hidden, whitespace is detected
344 from .spec.
345 Use --strict/-S option to ask rdopkg to return an exit status. By
347 default, 0 is returned.
348 Example:
350 rdopkg reqcheck
352 rdopkg reqcheck -s
353 Override file
355 There are instances when you need to ignore some python modules from
357 being reqcheck'ed, or you might need to change the module version that
358 is in the requirements.txt file in order to successfully pass the
359 reqcheck operation. Some examples of these are: - a python module which
360 has been added in the requirements file but not yet packaged in RDO.
361 This module should be ignored during the packaging time period. - a
362 python module which has been updated in the requirements file but not
363 yet packaged in RDO. This module version should be replaced with the
364 one that is specified in the .spec file. - an obsolete python module
365 which still lives in the requirement file, and not in .spec file, is
366 tagged as MISSING in reqcheck output. It’s a false positive. The module
367 should be removed upstream, in the meantime, we should ignore it. -
368 when a module RPM has several Provide for the same subpackage. As
369 pymod2pkg currently only suports one py3pkg name for the translation,
370 this can bring you to have a missing module message displayed, even
371 though the module is present with another name (e.g PyYAML module)
372 So, there are only two operations: 1. replacing a version of a python
374 module; 2. ignoring (i.e deleting) a python module;
375 The adding operation is not handled. The packages that are in .spec
377 file and not in requirements file are tagged as EXCESS during reqcheck,
378 which does not make the reqcheck operation fail. That’s why adding
379 python module in the requirements file before reqcheck is not relevant.
380 To ignore python module or overwrite version during reqcheck you need
382 to use --override/-O option, and provides a YAML file, see below the
383 format:
384 ---
386 packages:
387 all:
388 - name: "python-yaml"
389 - name: "python-pbr"
390 version: ""
391 openstack-murano:
392 - name: "python-alembic"
393 version: ">= 0.9.6"
394 - name: "python-pbr"
395 version: ">= 2.0.0"
396 The first level describes the package name we want to override (e.g
398 openstack-murano). The second level includes the configuration of
399 python modules we want to replace or ignore.
400 The keyword all is reserved in first level. The configuration (list of
402 python modules) associated to all is applied on all packages. This can
403 be interesting when a rule is spotted several times (e.g PyYAML), there
404 is a high chance that this rule might be applied everytime. You declare
405 it globally in all and there is no need anymore to explicitly specify
406 it in a package configuration. However, if a rule with the same python
407 module name is found in specific package (e.g openstack-murano) and all
408 configuration, then the rule in the specific package configuration is
409 picked up. Note: all configuration should be used carefully as the
410 associated rules are applied globally.
411 In this example, for the package named openstack-murano: - rdopkg will
413 replace alembic with alembic>=0.9.6 in the requirements file before
414 comparing to the alembic Requires version in the .spec file. - there is
415 no version attribute for python-yaml which is in all configuration, so
416 it will be ignored during the reqcheck. The behavior is the same when
417 version attribute has an empty value. - python-pbr with empty version
418 in all configuration means it is ignored during reqcheck. But, as it
419 exists a rule with the same python module name which lives in
420 openstack-murano, python-pbr>=2.0.0 is picked up during reqcheck.
421 Note: This option does not actually write to disk when replacing or
423 ignoring python modules in the requirements file. reqcheck loads this
424 file on memory stream, then the option --override replaces or ignores
425 the modules provided in the YAML file in the stream. Then, this stream
426 is compared against the .spec file.
427 Example:
429 rdopkg reqcheck --override override-file.yml
431 ACTION: reqdiff
433 Show pretty diff of requirements.txt.
434 See rdopkg-adv-requirements(7) for a complete example of reqdiff and
436 other requirements management actions.
437 Use this to see how requirements changed between versions.
439 See diff between current and latest upstream version (automagic):
441 rdopkg reqdiff
443 See diff between current and specified version:
445 rdopkg reqdiff 2015.1
447 See diff between two supplied versions:
449 rdopkg reqdiff 2015.1 2015.2
451 ACTION: kojibuild
453 Build the package in koji.
454 Flow:
456 • Run equivalent of fedpkg build using disgusting fedpkg python
458 module.
459 • Watch the build.
461 Example:
463 rdopkg kojibuild
465 ACTION: amend
467 Amend last git commit with current dist-git changes and (re)generate
468 the commit message from %changelog.
469 This simple action is equivalent to running
471 git commit -a --amend -m "$AUTOMAGIC_COMMIT_MESSAGE"
473 See AUTOMAGIC → commit message for more information about the generated
475 commit message.
476 ACTION: squash
478 Squash last git commit into previous one. The commit message of the
479 previous commit is used.
480 This simple action is a shortcut for
482 git reset --soft HEAD~
484 git commit --amend --no-edit
485 This is useful for squashing commits created by lower level actions
487 such as update-patches.
488 ACTION: get-sources
490 Download package source archive.
491 Currently, Source0 from .spec file is downloaded.
493 ACTION: info
495 Show information about RDO packaging.
496 Use this command to find out about:
498 • currently supported RDO OpenStack releases
500 • which distros are supported for each release
502 • what branch to build from
504 • what build system to build in
506 • supported packages
508 • various repositories tied to a package
510 • package maintainers
512 This command is a human interface to rdoinfo.
514 Releases/dists/branches overview:
516 rdopkg info
518 Detailed information about a package:
520 rdopkg info novaclient
522 Filter packages by maintainers:
524 rdopkg info maintainers:jruzicka
526 Reverse filtering packages by not containing tag:
528 rdopkg info tags:~newton
530 ACTION: release
532 Show information about RDO releases.
534 This command can be used to find out about:
536 * display list of all currently developed releases
538 * see their status, branch, repos, source branch
539 * filter information about single release
540 * print releases in specified phase
541 *To display info about all releases:
543 rdopkg release
545 *Show only specific release:
547 rdopkg release -r zed
549 *Print releases in development:
551 rdopkg release -s development
553 Currently used statuses are: development, maintained, extended_maintenance.
555 *Show releases that have repos for CentOS Stream 9:
557 rdopkg release -R el9s
559 ACTION: info-tags-diff
562 Show rdoinfo tag changes.
564 rdopkg info-tags-diff RDOINFODIR
566 will show per-package new/changed tags in rdoinfo between HEAD~..HEAD.
568 For an existing or new package, a list of changed tags is returned
570 Example:
572 $ rdopkg info-tags-diff ~/.rdopkg/rdoinfo
574 openstack-changed ['newton-uc', 'newton']
575 openstack-new-pkg ['under-review']
576 This is an interface to rdopkg.actionmods.rdoinfo:tags_diff().
578 ACTION: attr-tags-diff
580 Show rdoinfo attribute changes.
581 rdopkg info-attr-diff RDOINFODIR ATTRNAME [INFO_FILE]
583 will show per-package new/changed attributes in rdoinfo between
585 HEAD~..HEAD.
586 For an existing or new package, a list of changes in the attribute is
588 returned
589 Example:
591 $ rdopkg info-attr-diff ~/.rdopkg/rdoinfo upstream
593 openstack-changed 'https://github.com/openstack-changed-repo'
594 openstack-new-pkg 'https://github.com/openstack-new-pkg'
595 If ATTRNAME is not used anywhere in the distroinfo repository, or there
597 are no changes in the attributes, the command will return "No attribute
598 changes detected." and exit with RC=0.
599 You can optionally pass INFO_FILE as a parameter to define the
601 top-level YAML file to use.
602 This is an interface to rdopkg.actionmods.rdoinfo:attr_diff().
604 ACTION: findpkg
606 Find and show the single best matching package in rdoinfo.
607 This command produces the same output as rdopkg info but
609 • smart search is performed on package name, project name and
611 upstream URL
612 • only a single matching package is shown
614 Use -s/--strict to disable magic substring search and only match whole
616 fields.
617 This command is a human interface to distroinfo.query.find_package()
619 function.
620 Examples of usage:
622 rdopkg findpkg nova
624 rdopkg findpkg -s openstack-nova
625 rdopkg findpkg git://git.openstack.org/openstack/nova
626 rdopkg findpkg openstack/nova
627 rdopkg findpkg novacli
628 ACTION: conf
630 Display rdopkg’s local configuration.
631 This command prints the default configuration that ships with rdopkg
633 out of the box. You can override the individual settings here by using
634 .py files in the configuration directories.
635 Store your per-user configuration in ~/.rdopkg/conf.d/*.py, or store
637 system-wide configuration in /etc/rdopkg.d/*.py.
638 ACTION: tag-patches
640 Tag the local -patches branch with the package’s Name-Version-Release.
641 Since the -patches branch can change over time, including rebases,
643 rewrites, etc, we need a mechanism to keep historical records of what
644 the -patches branch looked like over time. Tagging the -patches branch
645 for each new NVR will maintain Git references to each snapshot of the
646 particular patches that went into each build.
647 To look at the -patches branch for an old build, you can simply "git
649 checkout name-version-release" for that build and get an exact
650 representation of the Git tree for that build.
651 If a previous tag exists with this name, rdopkg will exit with an error
653 unless you use the --force option to overwrite the existing tag with
654 this name.
655 You can automatically push the new tag with the --push option. It’s a
657 good idea to create and push the tag after every successful build.
658
660 Instead of requiring project config files or endless lists of command
661 line arguments, rdopkg tries to guess all the necessary variables.
662 patches branch
664 update-patches is a core lower level action for updating the dist-git
665 .spec file with patches from associated patches branch. rdopkg tries
666 hard to detect the patches branch automagically, it’s usually
667 $BRANCH-patches for $BRANCH dist-git but one patches branch per
668 multiple dist-gits is also supported.
669 Best illustrated by example, the following are all valid patches
671 branches for rhos-5.0-rhel-7 dist-git and they’re searched in that
672 order:
673 • rhos-5.0-rhel-7-patches
675 • rhos-5.0-rhel-patches
677 • rhos-5.0-patches ←-- preferred for RHOSP
679 • rhos-patches
681 Use rdopkg pkgenv to check detected patches branch.
683 You can specify remote patches branch by -p/--patches-branch action
685 parameter for actions that use it, such as patch and new-version.
686 You may explicitly set the name of your patches remote and patches
688 branch in your git configuration using the
689 rdopkg.<branch>.patches-remote and rdopkg.<branch>.patches-branch
690 options. For example, if you are working on a dist-git branch named
691 rhel-7.4 and you want to use rhel-7-patches for your patches branch,
692 you would run:
693 git config rdopkg.rhel-7.4.patches-branch rhel-7-patches
695 patches base
697 rdopkg calculates the git tag on which you are applying patches from
698 the Version tag in your .spec file. If your .spec file contains a macro
699 named milestone, the value of this macro will be appended to the
700 version. That is, if your spec file has:
701 Version: 2014.2.3
703 Then rdopkg will use 2014.2.3 as the base. If instead your .spec file
705 has:
706 %global milestone rc2
708 Version: 2014.2.3
710 Then rdopkg will use 2014.2.3rc2 as the base.
712 In older versions of rdopkg, it was necessary to explicitly set the
714 patch base using a special patches_base comment in your spec file. This
715 is now optional behavior (the patches base is calculated
716 automatically), but you can use this if you need to override the
717 automatic behavior.
718 The most common use of patches_base is to specify number of patches on
720 top of patches base (which defaults to spec Version) to skip:
721 # patches_base=+2
723 You can set an arbitrary git revision as a patches base:
725 # patches_base=1.2.3+2
727 You shouldn’t need to modify this by hand (other than perhaps the
729 number of skipped patches) as rdopkg manages patches_base as needed.
730 patches_ignore
732 update-patches also supports filtering out patches based on matching a
733 regex provided by a magic #patches_ignore comment in the spec file.
734 This is useful, for example, in case the patches branch contains
735 changes that are related to the CI/code review infra, that are useful
736 to keep around but don’t need to end up in the RPM.
737 For example, if you add the following comments in your package’s .spec
739 file:
740 # patches_base=10.2.5
742 # patches_ignore=DROP-IN-RPM
743 then rdopkg will not create .patch files for any commits that have
745 "DROP-IN-RPM" in the Git commit log’s subject line.
746 Note: these lines should be directly above any Patch000X lines in your
748 .spec file.
749 Release bumping
751 rdopkg fix and rdopkg patch bump the Release tag in .spec file.
752 By default, last numeric only part of Release is bumped:
754 1.1.1.a.b.c -> 1.1.2.a.b.c
756 You can override this using -R/--release-bump-index argument which
758 expects MAJOR/MINOR/PATCH or integer index of release part to bump,
759 starting at 1 from the left:
760 -R 1 / -R MAJOR: 1.1.1 -> 2.1.1
762 -R 2 / -R MINOR: 1.1.1 -> 1.2.1
763 -R 3 / -R PATCH: 1.1.1 -> 1.1.2
764 -R 4: 1.1.1.1 -> 1.1.1.2
765 -R 5: 1.1.1.1.1 -> 1.1.1.1.2
766 ...
767 DLRN 0.date.hash and 0.1.date.hash formats are detected and default to
769 bumping 2nd Release part (-R 2).
770 commit message
772 Commit messages created by rdopkg are generated from .spec file Name,
773 Version and Release (NVR) as well as last %changelog entry.
774 All rdopkg actions that modify distgit use following format:
776 package-name-1.2.3-4
778 Changelog:
780 - Doom the World (rhbz#111111)
781 - Fix Impending Doom support
782 - Save the World (rhbz#222222)
783 Resolves: rhbz#111111
785 Resolves: rhbz#222222
786 Change-Id: deadbeedeadbeedeadbeedeadbeedeadbeedeadbee
787 For each (rhbz#XYZ) mentioned in latest %changelog entry, Resolves:
789 rhbz#XYZ line is appended to commit message as required by RHOSP
790 workflow.
791 protip: To (re)generate nice commit message after modifying .spec file,
793 use rdopkg amend (see ACTION: amend above).
794 For example, following %changelog entry:
796 %changelog
798 * Tue Feb 11 2014 Jakub Ruzicka <jruzicka@redhat.com> 0.5.0-1
799 - Update to upstream 0.5.0
800 - Fix evil Bug of Doom (rhbz#123456)
801 will generate following commit message:
803 package-name-0.5.0-1
805 Changelog:
807 - Update to upstream 0.5.0
808 - Fix evil Bug of Doom (rhbz#123456)
809 Resolves: rhbz#123456
811 rdoinfo
813 rdoinfo is a special utility repository with RDO metadata:
814 https://github.com/redhat-openstack/rdoinfo
816 rdopkg uses rdoinfo to
818 • detect release/dist from branch name
820 • check valid RDO updates
822 • query packages from RDO/distribution repos
824 and more.
826 You can view the rdoinfo metada using rdopkg info.
828
830 rdopkg-adv-new-version(7), rdopkg-adv-requirements(7),
831 rdopkg-feature-pkgenv(7), rdopkg-feature-fix(7),
832 rdopkg-feature-patch(7), rdopkg-feature-new-version(7),
833 rdopkg-feature-actions(7)
834
836 rdopkg is maintained by Jakub Ruzicka <jruzicka@redhat.com>.
837 Bugs are tracked as github Issues:
839 https://github.com/softwarefactory-project/rdopkg/issues
841 To report a new bug:
843 https://github.com/softwarefactory-project/rdopkg/issues/new
845 10/04/2023 RDOPKG(1)