1RDOPKG(1) RDOPKG(1)
2
6 rdopkg - RDO packaging tool
7
9 rdopkg <action> <arg>...
10 rdopkg -c
12 rdopkg -h
14
16 rdopkg is a tool for automating RDO/RHOSP packaging tasks, such as
17 introducing patches, updating packages to new versions and submitting
18 packages to RDO.
19 Run rdopkg -h to get available actions.
21 persistence
23 rdopkg provides multi-step actions where each step is (should be)
24 idempotent so if something fails along the way or human interaction is
25 required, rdopkg drops to shell, lets you fix the problem, and then
26 continue by running rdopkg --continue (rdopkg -c).
27 The state is stored in a file named .rdopkg.json in the current
29 directory. The last stored action can be inspected (rdopkg status),
30 continued (rdopkg --continue) or aborted (rdopkg --abort). rdopkg will
31 refuse to perform a new multi-step action if a state file is present.
32
34 Important actions diagram
35 +---------------------+
38 / WHAT DO YOU NEED? /
39 +-----+----------+----+
40 |
41 |
42 |
43 build a new package
44 |
45 v
46 +---------------------+
47 / what is changing? /
48 +----+-----+-----+----+
49 | | |
50 +------+ | +--------+
51 | | |
52 just update introduce update to new
53 .spec patch(es) upstream version
54 | | |
55 | | |
56 v v v
57 +-----+ +-------+ +-------------+
58 | fix | | patch | | new-version |
59 +-----+ +-------+ +-------------+
60 ACTION: fix
62 Apply changes to the .spec file.
63 Flow:
65 · Bump Release, prepare a new %changelog entry header.
67 · Drop to shell, let user edit the .spec file.
69 · After running rdopkg, ensure description was added to %changelog
71 and commit changes in a new commit.
72 Example:
74 cd python-novaclient
76 rdopkg fix
77 vim python-novaclient.spec
78 rdopkg -c
79 More use cases can be found in rdopkg-feature-fix(7).
81 ACTION: patch
83 Introduce new patches to the package.
84 This action works with the patches branch (see AUTOMAGIC → patches
86 branch).
87 By default, rdopkg resets the local patches branch to the remote
89 patches branch. You can skip this with -l/--local-patches to directly
90 use the local patches branch as is.
91 Don’t forget to git fetch before running the action.
93 After running rdopkg patch, the new commit will contain the changes if
95 there are any.
96 You can use -C/--changelog option to select how rdopkg detects new/old
98 patches and changelog message:
99 · detect: detect new/old patches using commit hash/subject (default)
101 · count: count old and new patches (doesn’t work for simultaneous
103 removal & addition of patches and rewriting -patches history)
104 · plain: just use generic "- Update patches" message
106 Or you can use -B/--no-bump to skip Release bump and changelog
108 generation and only update patch files and their references in the
109 .spec file. This is useful when you only need to align distgit with the
110 patches branch.
111 Use --amend to amend previous git commit with the changes and
113 autogenerated commit message instead of creating a new one. This is
114 very convenient when modifying distgit commits.
115 Please see AUTOMAGIC for additional information about using magic
117 patches_base and patches_ignore comments in your .spec file to
118 influcence patches generation.
119 Flow:
121 · Unless -l/--local-patches was used, reset the local patches branch
123 to the remote patches branch.
124 · Update patch files from local patches branch using git
126 format-patch.
127 · Update .spec file with correct patch files references.
129 · Unless ‘-B/--no-bump` was used, update .spec file: bump Release,
131 create new %changelog entry with new patches’ titles depending on
132 -C/--changelog option.
133 · If a %global commit asdf1234 macro declaration is present, rewrite
135 it with the current sha1 of the patches branch. (This makes the
136 sha1 value available during your package’s build process. You can
137 use this to build your program so that "mycoolprogram --version"
138 could display the sha1 to users.)
139 · Create new commit (or amend previous one with -a/--amend) with the
141 changes using %changelog to generate commit message if available.
142 · Display the diff.
144 Example:
146 rdopkg patch
148 rdopkg patch -lBa
149 More use cases can be found in rdopkg-feature-patch(7).
151 ACTION: update-patches
153 An alias for:
154 rdopkg patch --local-patches --no-bump
156 in the spirit of the ancient update-patches.sh script.
158 See ACTION: PATCH above.
160 ACTION: new-version
162 Update package to new upstream version.
163 This action works with the patches branch (see AUTOMAGIC → patches
165 branch). After a successful rebase, rdopkg will offer to push the
166 rebased patches branch.
167 Required new-version argument is a new version to rebase on, presumably
169 a git version tag.
170 Don’t forget to git fetch --all before running the action.
172 You can use the -N/--new-sources or -n/--no-new-sources options to
174 control whether new-version will run fedpkg new-sources (rhpkg
175 new-sources on Red Hat downstream products). By default, rdopkg will
176 automatically enable this step in following scenarios:
177 · Fedora distgit detected from origin git remote
179 (pkgs.fedoraproject.org)
180 · RH distgit detected from git branch (rhos-*, rh-*, ceph-*,
182 rhscon-*)
183 Otherwise, fedpkg new-sources is disabled (-n).
185 After running rdopkg new-version, a new commit will contain the
187 changes.
188 To only update .spec without touching patches branch, -b/--bump-only
190 can be used. Along with -n/--no-new-sources this enables local-only
191 operations, much like rpmdev-bumpspec:
192 rdopkg -bn
194 To note particular bugs in the changelog, use the -B/--bug option.
196 rdopkg will append the supplied string to changelog in brackets. For
197 example:
198 rdopkg new-version --bug rhbz#1234,rhbz#5678
200 will result in following %changelog line:
202 Update to 1.1.1 (rhbz#1234,rhbz#5678)
204 and corresponding Resolves: lines in commit message.
206 Flow:
208 · Show changes between the previous version and the current one,
210 especially modifications to requirements.txt.
211 · Reset the local patches branch to the remote patches branch
213 · Rebase the local patches branch on $NEW_VERSION tag.
215 · Update .spec file: set Version, Release and patches_base to
217 appropriate values and create a new %changelog entry.
218 · Download source tarball.
220 · Run fedpkg new-sources (rhpkg new-sources).
222 · Update patches from the local patches branch.
224 · Display the diff.
226 Example:
228 cd python-novaclient
230 git fetch --all
231 rdopkg new-version 2.15.0
232 # rebase failed, manually fix using git
233 rdopkg -c
234 More use cases can be found in rdopkg-feature-new-version(7).
236 ACTION: lint
238 Run checks for errors in current distgit.
239 Available checks selectable with --lint-checks:
241 · sanity: internal rdopkg sanity checks on the .spec
243 · rpmlint: run rpmlint tool on the .spec
245 · all: run all available checks (default)
247 Available error levels selectable with --error-level affect the exit
249 code:
250 · E: exit with code 23 when linting error is found (default)
252 · W: exit with code 23 when linttng error or warning is found
254 · -: only print errors/warnings, always returns 0
256 Most of the time you probably want just:
258 rdopkg lint
260 Example of only running rpmlint with W error level:
262 rdopkg lint --lint-checks rpmlint --error-level W
264 ACTION: clone
266 Clone an RDO package distgit and setup remotes.
267 clone uses rdoinfo metadata to clone the specified RDO package distgit
269 and also setup relevant remotes to get you packaging quickly.
270 If your github username differs from your $USER, use -u/--review-user.
272 Example:
274 rdopkg clone -u github-user python-novaclient
276 cd python-novaclient
277 git remote -v
278 ACTION: query
280 Query RDO/distro repos for available package versions.
281 See rdopkg-adv-requirements(7) for complete example of query and other
283 requirements management actions.
284 This action uses repoquery to discover the latest package versions
286 available from RDO and other repos available on a supported
287 distibution.
288 See output of rdopkg info for supported releases and distros.
290 Query specific RELEASE/DIST:
292 rdopkg query kilo/el7 openstack-nova
294 Query all dists of a release and show what’s happening:
296 rdopkg query -v kilo openstack-nova
298 ACTION: reqquery
300 Query RDO/distro repos for versions defined in requirements.txt.
301 See rdopkg-adv-requirements(7) for a complete example of reqquery and
303 other requirements management actions.
304 This action essentially runs rdopkg query on every module/package
306 defined in requirements.txt and prints a colorful report to quickly
307 find unmet dependencies. It accepts the same RELEAESE/DIST filter as
308 rdopkg query.
309 Python module names listed in requirements.txt are mapped to package
311 names using the rdopkg.actionmods.pymod2pkg module.
312 Query requirements.txt from 2015.1 tag:
314 rdopkg reqquery -R 2015.1 kilo/el7
316 Query requirements.txt file:
318 rdopkg reqquery -r path/to/requirements.txt kilo/f21
320 Query .spec Requires (experimental):
322 rdopkg reqquery -s
324 Verbosely dump query results to a file and view them:
326 rdopkg reqquery -v -d
328 rdopkg reqquery -l
329 ACTION: reqcheck
331 Inspect requirements.txt vs .spec Requires.
332 See rdopkg-adv-requirements(7) for complete example of reqcheck and
334 other requirements management actions.
335 This action parses the current requirements.txt from git and checks
337 whether they’re met in the .spec file. A simple report is produced.
338 Python module names listed in requirements.txt are mapped to package
340 names using rdopkg.actionmods.pymod2pkg module.
341 Use --spec/-s option to output Requires: suitable for pasting into
343 .spec files. Version comparisons are hidden, whitespace is detected
344 from .spec.
345 Use --strict/-S option to ask rdopkg to return an exit status. By
347 default, 0 is returned.
348 Example:
350 rdopkg reqcheck
352 rdopkg reqcheck -s
353 Override file
355 There are instances when you need to ignore some python modules from
357 being reqcheck'ed, or you might need to change the module version that
358 is in the requirements.txt file in order to successfully pass the
359 reqcheck operation. Some examples of these are: - a python module which
360 has been added in the requirements file but not yet packaged in RDO.
361 This module should be ignored during the packaging time period. - a
362 python module which has been updated in the requirements file but not
363 yet packaged in RDO. This module version should be replaced with the
364 one that is specified in the .spec file. - an obsolete python module
365 which still lives in the requirement file, and not in .spec file, is
366 tagged as MISSING in reqcheck output. It’s a false positive. The module
367 should be removed upstream, in the meantime, we should ignore it. -
368 when a module RPM has several Provide for the same subpackage. As
369 pymod2pkg currently only suports one py3pkg name for the translation,
370 this can bring you to have a missing module message displayed, even
371 though the module is present with another name (e.g PyYAML module)
372 So, there are only two operations: 1. replacing a version of a python
374 module; 2. ignoring (i.e deleting) a python module;
375 The adding operation is not handled. The packages that are in .spec
377 file and not in requirements file are tagged as EXCESS during reqcheck,
378 which does not make the reqcheck operation fail. That’s why adding
379 python module in the requirements file before reqcheck is not relevant.
380 To ignore python module or overwrite version during reqcheck you need
382 to use --override/-O option, and provides a YAML file, see below the
383 format:
384 ---
386 packages:
387 all:
388 - name: "python-yaml"
389 - name: "python-pbr"
390 version: ""
391 openstack-murano:
392 - name: "python-alembic"
393 version: ">= 0.9.6"
394 - name: "python-pbr"
395 version: ">= 2.0.0"
396 The first level describes the package name we want to override (e.g
398 openstack-murano). The second level includes the configuration of
399 python modules we want to replace or ignore.
400 The keyword all is reserved in first level. The configuration (list of
402 python modules) associated to all is applied on all packages. This can
403 be interesting when a rule is spotted several times (e.g PyYAML), there
404 is a high chance that this rule might be applied everytime. You declare
405 it globally in all and there is no need anymore to explicitly specify
406 it in a package configuration. However, if a rule with the same python
407 module name is found in specific package (e.g openstack-murano) and all
408 configuration, then the rule in the specific package configuration is
409 picked up. Note: all configuration should be used carefully as the
410 associated rules are applied globally.
411 In this example, for the package named openstack-murano: - rdopkg will
413 replace alembic with alembic>=0.9.6 in the requirements file before
414 comparing to the alembic Requires version in the .spec file. - there is
415 no version attribute for python-yaml which is in all configuration, so
416 it will be ignored during the reqcheck. The behavior is the same when
417 version attribute has an empty value. - python-pbr with empty version
418 in all configuration means it is ignored during reqcheck. But, as it
419 exists a rule with the same python module name which lives in
420 openstack-murano, python-pbr>=2.0.0 is picked up during reqcheck.
421 Note: This option does not actually write to disk when replacing or
423 ignoring python modules in the requirements file. reqcheck loads this
424 file on memory stream, then the option --override replaces or ignores
425 the modules provided in the YAML file in the stream. Then, this stream
426 is compared against the .spec file.
427 Example:
429 rdopkg reqcheck --override override-file.yml
431 ACTION: reqdiff
433 Show pretty diff of requirements.txt.
434 See rdopkg-adv-requirements(7) for a complete example of reqdiff and
436 other requirements management actions.
437 Use this to see how requirements changed between versions.
439 See diff between current and latest upstream version (automagic):
441 rdopkg reqdiff
443 See diff between current and specified version:
445 rdopkg reqdiff 2015.1
447 See diff between two supplied versions:
449 rdopkg reqdiff 2015.1 2015.2
451 ACTION: kojibuild
453 Build the package in koji.
454 Flow:
456 · Run equivalent of fedpkg build using disgusting fedpkg python
458 module.
459 · Watch the build.
461 Example:
463 rdopkg kojibuild
465 ACTION: amend
467 Amend last git commit with current dist-git changes and (re)generate
468 the commit message from %changelog.
469 This simple action is equivalent to running
471 git commit -a --amend -m "$AUTOMAGIC_COMMIT_MESSAGE"
473 See AUTOMAGIC → commit message for more information about the generated
475 commit message.
476 ACTION: squash
478 Squash last git commit into previous one. The commit message of the
479 previous commit is used.
480 This simple action is a shortcut for
482 git reset --soft HEAD~
484 git commit --amend --no-edit
485 This is useful for squashing commits created by lower level actions
487 such as update-patches.
488 ACTION: get-sources
490 Download package source archive.
491 Currently, Source0 from .spec file is downloaded.
493 ACTION: info
495 Show information about RDO packaging.
496 Use this command to find out about:
498 · currently supported RDO OpenStack releases
500 · which distros are supported for each release
502 · what branch to build from
504 · what build system to build in
506 · supported packages
508 · various repositories tied to a package
510 · package maintainers
512 This command is a human interface to rdoinfo.
514 Releases/dists/branches overview:
516 rdopkg info
518 Detailed information about a package:
520 rdopkg info novaclient
522 Filter packages by maintainers:
524 rdopkg info maintainers:jruzicka
526 ACTION: info-tags-diff
528 Show rdoinfo tag changes.
529 rdopkg info-tags-diff RDOINFODIR
531 will show per-package new/changed tags in rdoinfo between HEAD~..HEAD.
533 For an existing or new package, a list of changed tags is returned
535 Example:
537 $ rdopkg info-tags-diff ~/.rdopkg/rdoinfo
539 openstack-changed ['newton-uc', 'newton']
540 openstack-new-pkg ['under-review']
541 This is an interface to rdopkg.actionmods.rdoinfo:tags_diff().
543 ACTION: findpkg
545 Find and show the single best matching package in rdoinfo.
546 This command produces the same output as rdopkg info but
548 · smart search is performed on package name, project name and
550 upstream URL
551 · only a single matching package is shown
553 Use -s/--strict to disable magic substring search and only match whole
555 fields.
556 This command is a human interface to distroinfo.query.find_package()
558 function.
559 Examples of usage:
561 rdopkg findpkg nova
563 rdopkg findpkg -s openstack-nova
564 rdopkg findpkg git://git.openstack.org/openstack/nova
565 rdopkg findpkg openstack/nova
566 rdopkg findpkg novacli
567 ACTION: conf
569 Display rdopkg’s local configuration.
570 This command prints the default configuration that ships with rdopkg
572 out of the box. You can override the individual settings here by using
573 .py files in the configuration directories.
574 Store your per-user configuration in ~/.rdopkg/conf.d/*.py, or store
576 system-wide configuration in /etc/rdopkg.d/*.py.
577 ACTION: tag-patches
579 Tag the local -patches branch with the package’s Name-Version-Release.
580 Since the -patches branch can change over time, including rebases,
582 rewrites, etc, we need a mechanism to keep historical records of what
583 the -patches branch looked like over time. Tagging the -patches branch
584 for each new NVR will maintain Git references to each snapshot of the
585 particular patches that went into each build.
586 To look at the -patches branch for an old build, you can simply "git
588 checkout name-version-release" for that build and get an exact
589 representation of the Git tree for that build.
590 If a previous tag exists with this name, rdopkg will exit with an error
592 unless you use the --force option to overwrite the existing tag with
593 this name.
594 You can automatically push the new tag with the --push option. It’s a
596 good idea to create and push the tag after every successful build.
597
599 Instead of requiring project config files or endless lists of command
600 line arguments, rdopkg tries to guess all the necessary variables.
601 patches branch
603 update-patches is a core lower level action for updating the dist-git
604 .spec file with patches from associated patches branch. rdopkg tries
605 hard to detect the patches branch automagically, it’s usually
606 $BRANCH-patches for $BRANCH dist-git but one patches branch per
607 multiple dist-gits is also supported.
608 Best illustrated by example, the following are all valid patches
610 branches for rhos-5.0-rhel-7 dist-git and they’re searched in that
611 order:
612 · rhos-5.0-rhel-7-patches
614 · rhos-5.0-rhel-patches
616 · rhos-5.0-patches ←-- preferred for RHOSP
618 · rhos-patches
620 Use rdopkg pkgenv to check detected patches branch.
622 You can specify remote patches branch by -p/--patches-branch action
624 parameter for actions that use it, such as patch and new-version.
625 You may explicitly set the name of your patches remote and patches
627 branch in your git configuration using the
628 rdopkg.<branch>.patches-remote and rdopkg.<branch>.patches-branch
629 options. For example, if you are working on a dist-git branch named
630 rhel-7.4 and you want to use rhel-7-patches for your patches branch,
631 you would run:
632 git config rdopkg.rhel-7.4.patches-branch rhel-7-patches
634 patches base
636 rdopkg calculates the git tag on which you are applying patches from
637 the Version tag in your .spec file. If your .spec file contains a macro
638 named milestone, the value of this macro will be appended to the
639 version. That is, if your spec file has:
640 Version: 2014.2.3
642 Then rdopkg will use 2014.2.3 as the base. If instead your .spec file
644 has:
645 %global milestone rc2
647 Version: 2014.2.3
649 Then rdopkg will use 2014.2.3rc2 as the base.
651 In older versions of rdopkg, it was necessary to explicitly set the
653 patch base using a special patches_base comment in your spec file. This
654 is now optional behavior (the patches base is calculated
655 automatically), but you can use this if you need to override the
656 automatic behavior.
657 The most common use of patches_base is to specify number of patches on
659 top of patches base (which defaults to spec Version) to skip:
660 # patches_base=+2
662 You can set an arbitrary git revision as a patches base:
664 # patches_base=1.2.3+2
666 You shouldn’t need to modify this by hand (other than perhaps the
668 number of skipped patches) as rdopkg manages patches_base as needed.
669 patches_ignore
671 update-patches also supports filtering out patches based on matching a
672 regex provided by a magic #patches_ignore comment in the spec file.
673 This is useful, for example, in case the patches branch contains
674 changes that are related to the CI/code review infra, that are useful
675 to keep around but don’t need to end up in the RPM.
676 For example, if you add the following comments in your package’s .spec
678 file:
679 # patches_base=10.2.5
681 # patches_ignore=DROP-IN-RPM
682 then rdopkg will not create .patch files for any commits that have
684 "DROP-IN-RPM" in the Git commit log’s subject line.
685 Note: these lines should be directly above any Patch000X lines in your
687 .spec file.
688 Release bumping
690 rdopkg fix and rdopkg patch bump the Release tag in .spec file.
691 By default, last numeric only part of Release is bumped:
693 1.1.1.a.b.c -> 1.1.2.a.b.c
695 You can override this using -R/--release-bump-index argument which
697 expects MAJOR/MINOR/PATCH or integer index of release part to bump,
698 starting at 1 from the left:
699 -R 1 / -R MAJOR: 1.1.1 -> 2.1.1
701 -R 2 / -R MINOR: 1.1.1 -> 1.2.1
702 -R 3 / -R PATCH: 1.1.1 -> 1.1.2
703 -R 4: 1.1.1.1 -> 1.1.1.2
704 -R 5: 1.1.1.1.1 -> 1.1.1.1.2
705 ...
706 DLRN 0.date.hash and 0.1.date.hash formats are detected and default to
708 bumping 2nd Release part (-R 2).
709 commit message
711 Commit messages created by rdopkg are generated from .spec file Name,
712 Version and Release (NVR) as well as last %changelog entry.
713 All rdopkg actions that modify distgit use following format:
715 package-name-1.2.3-4
717 Changelog:
719 - Doom the World (rhbz#111111)
720 - Fix Impending Doom support
721 - Save the World (rhbz#222222)
722 Resolves: rhbz#111111
724 Resolves: rhbz#222222
725 Change-Id: deadbeedeadbeedeadbeedeadbeedeadbeedeadbee
726 For each (rhbz#XYZ) mentioned in latest %changelog entry, Resolves:
728 rhbz#XYZ line is appended to commit message as required by RHOSP
729 workflow.
730 protip: To (re)generate nice commit message after modifying .spec file,
732 use rdopkg amend (see ACTION: amend above).
733 For example, following %changelog entry:
735 %changelog
737 * Tue Feb 11 2014 Jakub Ruzicka <jruzicka@redhat.com> 0.5.0-1
738 - Update to upstream 0.5.0
739 - Fix evil Bug of Doom (rhbz#123456)
740 will generate following commit message:
742 package-name-0.5.0-1
744 Changelog:
746 - Update to upstream 0.5.0
747 - Fix evil Bug of Doom (rhbz#123456)
748 Resolves: rhbz#123456
750 rdoinfo
752 rdoinfo is a special utility repository with RDO metadata:
753 https://github.com/redhat-openstack/rdoinfo
755 rdopkg uses rdoinfo to
757 · detect release/dist from branch name
759 · check valid RDO updates
761 · query packages from RDO/distribution repos
763 and more.
765 You can view the rdoinfo metada using rdopkg info.
767
769 rdopkg-adv-new-version(7), rdopkg-adv-requirements(7),
770 rdopkg-feature-pkgenv(7), rdopkg-feature-fix(7),
771 rdopkg-feature-patch(7), rdopkg-feature-new-version(7),
772 rdopkg-feature-actions(7)
773
775 rdopkg is maintained by Jakub Ruzicka <jruzicka@redhat.com>.
776 Bugs are tracked as github Issues:
778 https://github.com/softwarefactory-project/rdopkg/issues
780 To report a new bug:
782 https://github.com/softwarefactory-project/rdopkg/issues/new
784 08/21/2020 RDOPKG(1)