1dgit(7) dgit dgit(7)
2
6 dgit - principles of operation
7
9 dgit treats the Debian archive as a version control system, and bidi‐
10 rectionally gateways between the archive and git. The git view of the
11 package can contain the usual upstream git history, and will be aug‐
12 mented by commits representing uploads done by other developers not us‐
13 ing dgit. This git history is stored in a canonical location known as
14 dgit-repos which lives on a dedicated git server.
15 git branches suitable for use with dgit can be edited directly in git,
17 and used directly for building binary packages. They can be shared us‐
18 ing all conventional means for sharing git branches. It is not neces‐
19 sary to use dgit to work with dgitish git branches. However, dgit is
20 (usually) needed in order to convert to or from Debian-format source
21 packages.
22
24 dgit(1)
25 Reference manual and documentation catalogue.
26 dgit-*(7)
28 Tutorials and workflow guides. See dgit(1) for a list.
29
31 You may use any suitable git workflow with dgit, provided you satisfy
32 dgit's requirements:
33 dgit maintains a pseudo-remote called dgit, with one branch per suite.
35 This remote cannot be used with plain git.
36 The dgit-repos repository for each package contains one ref per suite
38 named refs/dgit/suite. These should be pushed to only by dgit. They
39 are fast forwarding. Each push on this branch corresponds to an upload
40 (or attempted upload).
41 However, it is perfectly fine to have other branches in dgit-repos;
43 normally the dgit-repos repo for the package will be accessible via the
44 remote name `origin'.
45 dgit push-* will also make signed tags called archive/debian/version
47 (with version encoded a la DEP-14) and push them to dgit-repos. These
48 are used at the server to authenticate pushes.
49 Uploads made by dgit contain an additional field Dgit in the source
51 package .dsc. (This is added by dgit push-*.) This specifies: a com‐
52 mit (an ancestor of the dgit/suite branch) whose tree is identical to
53 the unpacked source upload; the distro to which the upload was made; a
54 tag name which can be used to fetch the git commits; and a url to use
55 as a hint for the dgit git server for that distro.
56 Uploads not made by dgit are represented in git by commits which are
58 synthesised by dgit. The tree of each such commit corresponds to the
59 unpacked source; there is a commit with the contents, and a pseudo-
60 merge from last known upload - that is, from the contents of the
61 dgit/suite branch. Depending on the source package format, the con‐
62 tents commit may have a more complex structure, but ultimately it will
63 be a convergence of stubby branches from origin commits representing
64 the components of the source package.
65 dgit expects trees that it works with to have a dgit (pseudo) remote.
67 This refers to the dgit-created git view of the corresponding archive.
68 The dgit archive tracking view is synthesised locally, on demand, by
70 each copy of dgit. The tracking view is always a descendant of the
71 dgit-repos suite branch (if one exists), but may be ahead of it if up‐
72 loads have been done without dgit. The archive tracking view is always
73 fast forwarding within each suite.
74 dgit push-* can operate on any commit which is a descendant of the
76 suite tracking branch.
77 dgit does not make a systematic record of its imports of orig tar‐
79 ball(s). So it does not work by finding git tags or branches referring
80 to orig tarball(s). The orig tarballs are downloaded (by dgit clone)
81 into the parent directory, as with a traditional (non-gitish) dpkg-
82 source workflow. You need to retain these tarballs in the parent di‐
83 rectory for dgit build and dgit push-*. (They are not needed for
84 purely-git-based workflows.)
85 dgit repositories could be cloned with standard (git) methods. How‐
87 ever, the dgit repositories do not contain uploads not made with dgit.
88 And for sourceful builds / uploads the orig tarball(s) will need to be
89 present in the parent directory.
90 To a user looking at the archive, changes pushed in a simple NMU using
92 dgit look like reasonable changes made in an NMU: in a `3.0 (quilt)'
93 package the delta from the previous upload is recorded in new patch(es)
94 constructed by dpkg-source.
95
97 dgit can synthesize a combined view of several underlying suites. This
98 is requested by specifying, for suite, a comma-separated list:
99 mainsuite,subsuite...
101 This facility is available with dgit clone, fetch and pull, only.
103 dgit will fetch the same package from each specified underlying suite,
105 separately (as if with dgit fetch). dgit will then generate a pseu‐
106 domerge commit on the tracking branch remotes/dgit/dgit/suite which has
107 the tip of each of the underlying suites as an ancestor, and which con‐
108 tains the same as the suite which has the highest version of the pack‐
109 age.
110 The package must exist in mainsuite, but need not exist in the sub‐
112 suites.
113 If a specified subsuite starts with - then mainsuite is prepended.
115 So, for example, stable,-security means to look for the package in sta‐
117 ble, and stable-security, taking whichever is newer. If stable is cur‐
118 rently jessie, dgit clone would leave you on the branch
119 dgit/jessie,-security.
120 Combined suites are not supported by the dgit build operations. This
122 is because those options are intended for building for uploading source
123 packages, and look in the changelog to find the relevant suite. It
124 does not make sense to name a dgit-synthesised combined suite in a
125 changelog, or to try to upload to it.
126 When using this facility, it is important to always specify the same
128 suites in the same order: dgit will not make a coherent fast-forwarding
129 history view otherwise.
130 The history generated by this feature is not normally suitable for
132 merging back into upstreams, as it necessarily contains unattractive
133 pseudomerges.
134
136 Because the synthesis of the suite tracking branches is done locally
137 based only on the current archive state, it will not necessarily see
138 every upload not done with dgit. Also, different versions of dgit (or
139 the software it calls) might import the same .dscs differently (al‐
140 though we try to minimise this). As a consequence, the dgit tracking
141 views of the same suite, made by different instances of dgit, may vary.
142 They will have the same contents, but may have different history.
143 There is no uniform linkage between the tracking branches for different
145 suites. The Debian infrastructure does not do any automatic import of
146 uploads made without dgit. It would be possible for a distro's infra‐
147 structure to do this; in that case, different dgit client instances
148 would see exactly the same history.
149 There has been no bulk import of historical uploads into Debian's dgit
151 infrastructure. To do this it would be necessary to decide whether to
152 import existing vcs history (which might not be faithful to dgit's in‐
153 variants) or previous non-Dgit uploads (which would not provide a very
154 rich history).
155 git represents only file executability. git does not represent empty
157 directories, or any leaf objects other than plain files and symlinks.
158 The behaviour of Debian source package formats on objects with unusual
159 permissions is complicated. Some pathological Debian source packages
160 will no longer build if empty directories are pruned (or if other
161 things not reproduced by git are changed). Such sources cannot be
162 worked with properly in git, and therefore not with dgit either.
163
165 Distros which do not maintain a set of dgit history git repositories
166 can still be used in a read-only mode with dgit. Currently Ubuntu is
167 configured this way.
168
170 git has features which can automatically transform files as they are
171 being copied between the working tree and the git history. The at‐
172 tributes can be specified in the source tree itself, in .gitattributes.
173 See gitattributes(5).
174 These transformations are context-sensitive and not, in general, re‐
176 versible, so dgit operates on the principle that the dgit git history
177 contains the actual contents of the package. (When dgit is manipulat‐
178 ing a .dsc, it does so in a private area, where the transforming gitat‐
179 tributes are defused, to achieve this.)
180 If transforming gitattributes are used, they can cause trouble, because
182 the working tree files can differ from the git revision history (and
183 therefore from the source packages). dgit warns if it finds a .gitat‐
184 tributes file (in a package being fetched or imported), unless the
185 transforming gitattributes have been defused.
186 dgit clone and dgit setup-new-tree disable transforming gitattributes
188 by default, by creating a suitable .git/info/attributes. See dgit
189 setup-new-tree and dgit setup-gitattributes in dgit(1).
190 Note that dgit does not disable gitattributes unless they would actu‐
192 ally interfere with your work on dgit branches. In particular, gitat‐
193 tributes which affect git archive are not disabled, so .origs you gen‐
194 erate by hand can be wrong. You should consider using git-deborig (1)
195 which gets this right, suppressing the attributes.
196
198 If you are not the maintainer, you do not need to worry about the
199 source format of the package. You can just make changes as you like in
200 git. If the package is a `3.0 (quilt)' package, the patch stack will
201 usually not be represented in the git history.
202
204 Debian source package formats do not always faithfully reproduce
205 changes to executability. But dgit insists that the result of dgit
206 clone is identical (as far as git can represent - see Limitations,
207 above) to the result of dpkg-source -x.
208 So files that are executable in your git tree must be executable in the
210 result of dpkg-source -x (but often aren't). If a package has such
211 troublesome files, they have to be non-executable in dgit-compatible
212 git branches.
213
215 For a format `3.0 (quilt)' source package, dgit may have to make a com‐
216 mit on your current branch to contain metadata used by quilt and dpkg-
217 source.
218 This is because `3.0 (quilt)' source format represents the patch stack
220 as files in debian/patches/ actually inside the source tree. This
221 means that, taking the whole tree (as seen by git or ls) (i) dpkg-
222 source cannot represent certain trees, and (ii) packing up a tree in
223 `3.0 (quilt)' and then unpacking it does not always yield the same
224 tree.
225 dgit will automatically work around this for you when building and
227 pushing. The only thing you need to know is that dgit build, sbuild,
228 etc., may make new commits on your HEAD. If you're not a quilt user
229 this commit won't contain any changes to files you care about.
230 Simply committing to source files (whether in debian/ or not, but not
232 to patches) will result in a branch that dgit quilt-fixup can lin‐
233 earise. Other kinds of changes, including editing patches or merging,
234 cannot be handled this way.
235 You can explicitly request that dgit do just this fixup, by running
237 dgit quilt-fixup.
238 If you are a quilt user you need to know that dgit's git trees are
240 `patches applied packaging branches' and do not contain the .pc direc‐
241 tory (which is used by quilt to record which patches are applied). If
242 you want to manipulate the patch stack you probably want to be looking
243 at tools like git-debrebase, gbp pq, or git-dpm.
244 quilt fixup error messages
247 When dgit's quilt fixup fails, it prints messages like this:
248 dgit: base trees orig=5531f03d8456b702eab6 o+d/p=135338e9cc253cc85f84
250 dgit: quilt differences: src: == orig ## gitignores: == orig ##
251 dgit: quilt differences: HEAD ## o+d/p HEAD ## o+d/p
252 starting quiltify (multiple patches, linear mode)
253 dgit: error: quilt fixup cannot be linear. Stopped at:
255 dgit: 696c9bd5..84ae8f96: changed debian/patches/test-gitignore
256 orig is an import of the .orig tarballs dgit found, with the debian/
259 directory from your HEAD substituted. This is a git tree ob‐
260 ject, not a commit: you can pass its hash to git-diff but not
261 git-log.
262 o+d/p is another tree object, which is the same as orig but with the
265 patches from debian/patches applied.
266 HEAD is of course your own git HEAD.
269 quilt differences
272 shows whether each of the these trees differs from the others
273 (i) in upstream files excluding .gitignore files; (ii) in up‐
274 stream .gitignore files. == indicates equality; ## indicates
275 inequality.
276 dgit quilt-fixup --quilt=linear walks commits backwards from your HEAD
278 trying to construct a linear set of additional patches, starting at the
279 end. It hopes to eventually find an ancestor whose tree is identical
280 to o+d/p in all upstream files.
281 In the error message, 696c9bd5..84ae8f96 is the first commit child-par‐
283 ent edge which cannot sensibly be either ignored, or turned into a
284 patch in debian/patches. In this example, this is because it itself
285 changes files in debian/patches, indicating that something unusual is
286 going on and that continuing is not safe. But you might also see other
287 kinds of troublesome commit or edge.
288 Your appropriate response depends on the cause and the context. If you
290 have been freely merging your git branch and do not need need a pretty
291 linear patch queue, you can use --quilt=single or --quilt=smash.
292 (Don't use the single-debian-patch dpkg source format option; it has
293 strange properties.) If you want a pretty linear series, and this mes‐
294 sage is unexpected, it can mean that you have unwittingly committed
295 changes that are not representable by dpkg-source (such as some mode
296 changes). Or maybe you just forgot a necessary --quilt= option.
297 Finally, this problem can occur if you have provided Debian git tooling
299 such as git-debrebase, git-dpm or git-buildpackage with upstream git
300 commit(s) or tag(s) which are not 100% identical to your orig tar‐
301 ball(s).
302
304 When working with git branches intended for use with the `3.0 (quilt)'
305 source format dgit can automatically convert a suitable maintainer-pro‐
306 vided git branch (in one of a variety of formats) into a dgit branch.
307 When a splitting quilt mode is selected dgit build commands and dgit
309 push-* will, on each invocation, convert the user's HEAD into the dgit
310 view, so that it can be built and/or uploaded.
311 Split view mode can also be enabled explicitly with the --split-view
313 command line option and the .split-view access configuration key.
314 When split view is in operation, regardless of the quilt mode, any
316 dgit-generated pseudomerges and any quilt fixup commits will appear
317 only in the dgit view. dgit push-* will push the dgit view to the dgit
318 git server. The dgit view is always a descendant of the maintainer
319 view. dgit push-* will also make a maintainer view tag according to
320 DEP-14 and push that to the dgit git server.
321 Splitting quilt modes must be enabled explicitly (by the use of the ap‐
323 plicable command line options, subcommands, or configuration). This is
324 because it is not possible to reliably tell (for example) whether a git
325 tree for a dpkg-source `3.0 (quilt)' package is a patches-applied or
326 patches-unapplied tree.
327 Split view conversions are cached in the ref dgit-intern/quilt-cache.
329 This should not be manipulated directly.
330
332 This section is mainly of interest to maintainers who want to use dgit
333 with their existing git history for the Debian package.
334 Some developers like to have an extra-clean git tree which lacks files
336 which are normally found in source tarballs and therefore in Debian
337 source packages. For example, it is conventional to ship ./configure
338 in the source tarball, but some people prefer not to have it present in
339 the git view of their project.
340 dgit requires that the source package unpacks to exactly the same files
342 as are in the git commit on which dgit push-* operates. So if you just
343 try to dgit push-* directly from one of these extra-clean git branches,
344 it will fail.
345 As the maintainer you therefore have the following options:
347 • Delete the files from your git branches, and your Debian source
349 packages, and carry the deletion as a delta from upstream.
350 (With `3.0 (quilt)' this means representing the deletions as
351 patches. You may need to pass --include-removal to dpkg-source
352 --commit, or pass corresponding options to other tools.) This
353 can make the Debian source package less useful for people with‐
354 out Debian build infrastructure.
355 • Persuade upstream that the source code in their git history and
357 the source they ship as tarballs should be identical. Of course
358 simply removing the files from the tarball may make the tarball
359 hard for people to use.
360 One answer is to commit the (maybe autogenerated) files, perhaps
362 with some simple automation to deal with conflicts and spurious
363 changes. This has the advantage that someone who clones the git
364 repository finds the program just as easy to build as someone
365 who uses the tarball.
366 Of course it may also be that the differences are due to build system
368 bugs, which cause unintended files to end up in the source package.
369 dgit will notice this and complain. You may have to fix these bugs be‐
370 fore you can unify your existing git history with dgit's.
371
373 Some upstream tarballs contain build artifacts which upstream expects
374 some users not to want to rebuild (or indeed to find hard to rebuild),
375 but which in Debian we always rebuild.
376 Examples sometimes include crossbuild firmware binaries and documenta‐
378 tion. To avoid problems when building updated source packages (in par‐
379 ticular, to avoid trying to represent as changes in the source package
380 uninteresting or perhaps unrepresentable changes to such files) many
381 maintainers arrange for the package clean target to delete these files.
382 dpkg-source does not (with any of the commonly used source formats)
384 represent deletion of binaries (outside debian/) present in upstream.
385 Thus deleting such files in a dpkg-source working tree does not actu‐
386 ally result in them being deleted from the source package. Thus delet‐
387 ing the files in rules clean sweeps this problem under the rug.
388 However, git does always properly record file deletion. Since dgit's
390 principle is that the dgit git tree is the same of dpkg-source -x, that
391 means that a dgit-compatible git tree always contains these files.
392 For the non-maintainer, this can be observed in the following subopti‐
394 mal occurrences:
395 • The package clean target often deletes these files, making the
397 git tree dirty trying to build the source package, etc. This
398 can be fixed by using dgit -wg aka --clean=git, so that the
399 package clean target is never run.
400 • The package build modifies these files, so that builds make the
402 git tree dirty. This can be worked around by using `git reset
403 --hard' after each build (or at least before each commit or
404 push).
405 From the maintainer's point of view, the main consequence is that to
407 make a dgit-compatible git branch it is necessary to commit these files
408 to git. The maintainer has a few additional options for mitigation:
409 for example, it may be possible for the rules file to arrange to do the
410 build in a temporary area, which avoids updating the troublesome files;
411 they can then be left in the git tree without seeing trouble.
412
414 A related problem is other unexpected behaviour by a package's clean
415 target. If a package's rules modify files which are distributed in the
416 package, or simply forget to remove certain files, dgit will complain
417 that the tree is dirty.
418 Again, the solution is to use dgit -wg aka --clean=git, which instructs
420 dgit to use git clean instead of the package's build target, along with
421 perhaps git reset --hard before each build.
422 This is 100% reliable, but has the downside that if you forget to git
424 add or to commit, and then use dgit -wg or git reset --hard, your
425 changes may be lost.
426Debian Project dgit(7)