1dgit(7) dgit dgit(7)
2
6 dgit - principles of operation
7
9 dgit treats the Debian archive as a version control system, and bidi‐
10 rectionally gateways between the archive and git. The git view of the
11 package can contain the usual upstream git history, and will be aug‐
12 mented by commits representing uploads done by other developers not
13 using dgit. This git history is stored in a canonical location known
14 as dgit-repos which lives on a dedicated git server.
15 git branches suitable for use with dgit can be edited directly in git,
17 and used directly for building binary packages. They can be shared
18 using all conventional means for sharing git branches. It is not nec‐
19 essary to use dgit to work with dgitish git branches. However, dgit is
20 (usually) needed in order to convert to or from Debian-format source
21 packages.
22
24 dgit(1)
25 Reference manual and documentation catalogue.
26 dgit-*(7)
28 Tutorials and workflow guides. See dgit(1) for a list.
29
31 You may use any suitable git workflow with dgit, provided you satisfy
32 dgit's requirements:
33 dgit maintains a pseudo-remote called dgit, with one branch per suite.
35 This remote cannot be used with plain git.
36 The dgit-repos repository for each package contains one ref per suite
38 named refs/dgit/suite. These should be pushed to only by dgit. They
39 are fast forwarding. Each push on this branch corresponds to an upload
40 (or attempted upload).
41 However, it is perfectly fine to have other branches in dgit-repos;
43 normally the dgit-repos repo for the package will be accessible via the
44 remote name `origin'.
45 dgit push will also make signed tags called archive/debian/version
47 (with version encoded a la DEP-14) and push them to dgit-repos. These
48 are used at the server to authenticate pushes.
49 Uploads made by dgit contain an additional field Dgit in the source
51 package .dsc. (This is added by dgit push.) This specifies: a commit
52 (an ancestor of the dgit/suite branch) whose tree is identical to the
53 unpacked source upload; the distro to which the upload was made; a tag
54 name which can be used to fetch the git commits; and a url to use as a
55 hint for the dgit git server for that distro.
56 Uploads not made by dgit are represented in git by commits which are
58 synthesised by dgit. The tree of each such commit corresponds to the
59 unpacked source; there is a commit with the contents, and a pseudo-
60 merge from last known upload - that is, from the contents of the
61 dgit/suite branch. Depending on the source package format, the con‐
62 tents commit may have a more complex structure, but ultimately it will
63 be a convergence of stubby branches from origin commits representing
64 the components of the source package.
65 dgit expects trees that it works with to have a dgit (pseudo) remote.
67 This refers to the dgit-created git view of the corresponding archive.
68 The dgit archive tracking view is synthesised locally, on demand, by
70 each copy of dgit. The tracking view is always a descendant of the
71 dgit-repos suite branch (if one exists), but may be ahead of it if
72 uploads have been done without dgit. The archive tracking view is
73 always fast forwarding within each suite.
74 dgit push can operate on any commit which is a descendant of the suite
76 tracking branch.
77 dgit does not make a systematic record of its imports of orig tar‐
79 ball(s). So it does not work by finding git tags or branches referring
80 to orig tarball(s). The orig tarballs are downloaded (by dgit clone)
81 into the parent directory, as with a traditional (non-gitish) dpkg-
82 source workflow. You need to retain these tarballs in the parent
83 directory for dgit build and dgit push. (They are not needed for
84 purely-git-based workflows.)
85 dgit repositories could be cloned with standard (git) methods. How‐
87 ever, the dgit repositories do not contain uploads not made with dgit.
88 And for sourceful builds / uploads the orig tarball(s) will need to be
89 present in the parent directory.
90 To a user looking at the archive, changes pushed in a simple NMU using
92 dgit look like reasonable changes made in an NMU: in a `3.0 (quilt)'
93 package the delta from the previous upload is recorded in new patch(es)
94 constructed by dpkg-source.
95
97 dgit can synthesize a combined view of several underlying suites. This
98 is requested by specifying, for suite, a comma-separated list:
99 mainsuite,subsuite...
101 This facility is available with dgit clone, fetch and pull, only.
103 dgit will fetch the same package from each specified underlying suite,
105 separately (as if with dgit fetch). dgit will then generate a pseu‐
106 domerge commit on the tracking branch remotes/dgit/dgit/suite which has
107 the tip of each of the underlying suites as an ancestor, and which con‐
108 tains the same as the suite which has the highest version of the pack‐
109 age.
110 The package must exist in mainsuite, but need not exist in the sub‐
112 suites.
113 If a specified subsuite starts with - then mainsuite is prepended.
115 So, for example, stable,-security means to look for the package in sta‐
117 ble, and stable-security, taking whichever is newer. If stable is cur‐
118 rently jessie, dgit clone would leave you on the branch
119 dgit/jessie,-security.
120 Combined suites are not supported by the dgit build operations. This
122 is because those options are intended for building for uploading source
123 packages, and look in the changelog to find the relevant suite. It
124 does not make sense to name a dgit-synthesised combined suite in a
125 changelog, or to try to upload to it.
126 When using this facility, it is important to always specify the same
128 suites in the same order: dgit will not make a coherent fast-forwarding
129 history view otherwise.
130 The history generated by this feature is not normally suitable for
132 merging back into upstreams, as it necessarily contains unattractive
133 pseudomerges.
134
136 Because the synthesis of the suite tracking branches is done locally
137 based only on the current archive state, it will not necessarily see
138 every upload not done with dgit. Also, different versions of dgit (or
139 the software it calls) might import the same .dscs differently
140 (although we try to minimise this). As a consequence, the dgit track‐
141 ing views of the same suite, made by different instances of dgit, may
142 vary. They will have the same contents, but may have different his‐
143 tory.
144 There is no uniform linkage between the tracking branches for different
146 suites. The Debian infrastructure does not do any automatic import of
147 uploads made without dgit. It would be possible for a distro's infra‐
148 structure to do this; in that case, different dgit client instances
149 would see exactly the same history.
150 There has been no bulk import of historical uploads into Debian's dgit
152 infrastructure. To do this it would be necessary to decide whether to
153 import existing vcs history (which might not be faithful to dgit's
154 invariants) or previous non-Dgit uploads (which would not provide a
155 very rich history).
156 git represents only file executability. git does not represent empty
158 directories, or any leaf objects other than plain files and symlinks.
159 The behaviour of Debian source package formats on objects with unusual
160 permissions is complicated. Some pathological Debian source packages
161 will no longer build if empty directories are pruned (or if other
162 things not reproduced by git are changed). Such sources cannot be
163 worked with properly in git, and therefore not with dgit either.
164
166 Distros which do not maintain a set of dgit history git repositories
167 can still be used in a read-only mode with dgit. Currently Ubuntu is
168 configured this way.
169
171 git has features which can automatically transform files as they are
172 being copied between the working tree and the git history. The
173 attributes can be specified in the source tree itself, in .gitat‐
174 tributes. See gitattributes(5).
175 These transformations are context-sensitive and not, in general,
177 reversible, so dgit operates on the principle that the dgit git history
178 contains the actual contents of the package. (When dgit is manipulat‐
179 ing a .dsc, it does so in a private area, where the transforming gitat‐
180 tributes are defused, to achieve this.)
181 If transforming gitattributes are used, they can cause trouble, because
183 the working tree files can differ from the git revision history (and
184 therefore from the source packages). dgit warns if it finds a .gitat‐
185 tributes file (in a package being fetched or imported), unless the
186 transforming gitattributes have been defused.
187 dgit clone and dgit setup-new-tree disable transforming gitattributes
189 by default, by creating a suitable .git/info/attributes. See dgit set‐
190 up-new-tree and dgit setup-gitattributes in dgit(1).
191 Note that dgit does not disable gitattributes unless they would actu‐
193 ally interfere with your work on dgit branches. In particular, gitat‐
194 tributes which affect git archive are not disabled, so .origs you gen‐
195 erate by hand can be wrong. You should consider using git-deborig (1)
196 which gets this right, suppressing the attributes.
197
199 If you are not the maintainer, you do not need to worry about the
200 source format of the package. You can just make changes as you like in
201 git. If the package is a `3.0 (quilt)' package, the patch stack will
202 usually not be represented in the git history.
203
205 Debian source package formats do not always faithfully reproduce
206 changes to executability. But dgit insists that the result of dgit
207 clone is identical (as far as git can represent - see Limitations,
208 above) to the result of dpkg-source -x.
209 So files that are executable in your git tree must be executable in the
211 result of dpkg-source -x (but often aren't). If a package has such
212 troublesome files, they have to be non-executable in dgit-compatible
213 git branches.
214
216 For a format `3.0 (quilt)' source package, dgit may have to make a com‐
217 mit on your current branch to contain metadata used by quilt and dpkg-
218 source.
219 This is because `3.0 (quilt)' source format represents the patch stack
221 as files in debian/patches/ actually inside the source tree. This
222 means that, taking the whole tree (as seen by git or ls) (i) dpkg-
223 source cannot represent certain trees, and (ii) packing up a tree in
224 `3.0 (quilt)' and then unpacking it does not always yield the same
225 tree.
226 dgit will automatically work around this for you when building and
228 pushing. The only thing you need to know is that dgit build, sbuild,
229 etc., may make new commits on your HEAD. If you're not a quilt user
230 this commit won't contain any changes to files you care about.
231 Simply committing to source files (whether in debian/ or not, but not
233 to patches) will result in a branch that dgit quilt-fixup can lin‐
234 earise. Other kinds of changes, including editing patches or merging,
235 cannot be handled this way.
236 You can explicitly request that dgit do just this fixup, by running
238 dgit quilt-fixup.
239 If you are a quilt user you need to know that dgit's git trees are
241 `patches applied packaging branches' and do not contain the .pc direc‐
242 tory (which is used by quilt to record which patches are applied). If
243 you want to manipulate the patch stack you probably want to be looking
244 at tools like git-debrebase, gbp pq, or git-dpm.
245 quilt fixup error messages
248 When dgit's quilt fixup fails, it prints messages like this:
249 dgit: base trees orig=5531f03d8456b702eab6 o+d/p=135338e9cc253cc85f84
251 dgit: quilt differences: src: == orig ## gitignores: == orig ##
252 dgit: quilt differences: HEAD ## o+d/p HEAD ## o+d/p
253 starting quiltify (multiple patches, linear mode)
254 dgit: error: quilt fixup cannot be linear. Stopped at:
256 dgit: 696c9bd5..84ae8f96: changed debian/patches/test-gitignore
257 orig is an import of the .orig tarballs dgit found, with the debian/
260 directory from your HEAD substituted. This is a git tree
261 object, not a commit: you can pass its hash to git-diff but not
262 git-log.
263 o+d/p is another tree object, which is the same as orig but with the
266 patches from debian/patches applied.
267 HEAD is of course your own git HEAD.
270 quilt differences
273 shows whether each of the these trees differs from the others
274 (i) in upstream files excluding .gitignore files; (ii) in
275 upstream .gitignore files. == indicates equality; ## indicates
276 inequality.
277 dgit quilt-fixup --quilt=linear walks commits backwards from your HEAD
279 trying to construct a linear set of additional patches, starting at the
280 end. It hopes to eventually find an ancestor whose tree is identical
281 to o+d/p in all upstream files.
282 In the error message, 696c9bd5..84ae8f96 is the first commit child-par‐
284 ent edge which cannot sensibly be either ignored, or turned into a
285 patch in debian/patches. In this example, this is because it itself
286 changes files in debian/patches, indicating that something unusual is
287 going on and that continuing is not safe. But you might also see other
288 kinds of troublesome commit or edge.
289 Your appropriate response depends on the cause and the context. If you
291 have been freely merging your git branch and do not need need a pretty
292 linear patch queue, you can use --quilt=smash (or use the 1.0 or sin‐
293 gle-debian-patch source formats; see dpkg-source(1).) If you want a
294 pretty linear series, and this message is unexpected, it can mean that
295 you have unwittingly committed changes that are not representable by
296 dpkg-source (such as some mode changes). Or maybe you just forgot a
297 necessary --quilt= option.
298 Finally, this problem can occur if you have provided Debian git tooling
300 such as git-debrebase, git-dpm or git-buildpackage with upstream git
301 commit(s) or tag(s) which are not 100% identical to your orig tar‐
302 ball(s).
303
305 When working with git branches intended for use with the `3.0 (quilt)'
306 source format dgit can automatically convert a suitable maintainer-pro‐
307 vided git branch (in one of a variety of formats) into a dgit branch.
308 When a splitting quilt mode is selected dgit build commands and dgit
310 push will, on each invocation, convert the user's HEAD into the dgit
311 view, so that it can be built and/or uploaded.
312 Split view mode can also be enabled explicitly with the --split-view
314 command line option and the .split-view access configuration key.
315 When split view is in operation, regardless of the quilt mode, any
317 dgit-generated pseudomerges and any quilt fixup commits will appear
318 only in the dgit view. dgit push will push the dgit view to the dgit
319 git server. The dgit view is always a descendant of the maintainer
320 view. dgit push will also make a maintainer view tag according to
321 DEP-14 and push that to the dgit git server.
322 Splitting quilt modes must be enabled explicitly (by the use of the
324 applicable command line options, subcommands, or configuration). This
325 is because it is not possible to reliably tell (for example) whether a
326 git tree for a dpkg-source `3.0 (quilt)' package is a patches-applied
327 or patches-unapplied tree.
328 Split view conversions are cached in the ref dgit-intern/quilt-cache.
330 This should not be manipulated directly.
331
333 This section is mainly of interest to maintainers who want to use dgit
334 with their existing git history for the Debian package.
335 Some developers like to have an extra-clean git tree which lacks files
337 which are normally found in source tarballs and therefore in Debian
338 source packages. For example, it is conventional to ship ./configure
339 in the source tarball, but some people prefer not to have it present in
340 the git view of their project.
341 dgit requires that the source package unpacks to exactly the same files
343 as are in the git commit on which dgit push operates. So if you just
344 try to dgit push directly from one of these extra-clean git branches,
345 it will fail.
346 As the maintainer you therefore have the following options:
348 · Delete the files from your git branches, and your Debian source
350 packages, and carry the deletion as a delta from upstream.
351 (With `3.0 (quilt)' this means representing the deletions as
352 patches. You may need to pass --include-removal to dpkg-source
353 --commit, or pass corresponding options to other tools.) This
354 can make the Debian source package less useful for people with‐
355 out Debian build infrastructure.
356 · Persuade upstream that the source code in their git history and
358 the source they ship as tarballs should be identical. Of course
359 simply removing the files from the tarball may make the tarball
360 hard for people to use.
361 One answer is to commit the (maybe autogenerated) files, perhaps
363 with some simple automation to deal with conflicts and spurious
364 changes. This has the advantage that someone who clones the git
365 repository finds the program just as easy to build as someone
366 who uses the tarball.
367 Of course it may also be that the differences are due to build system
369 bugs, which cause unintended files to end up in the source package.
370 dgit will notice this and complain. You may have to fix these bugs
371 before you can unify your existing git history with dgit's.
372
374 Some upstream tarballs contain build artifacts which upstream expects
375 some users not to want to rebuild (or indeed to find hard to rebuild),
376 but which in Debian we always rebuild.
377 Examples sometimes include crossbuild firmware binaries and documenta‐
379 tion. To avoid problems when building updated source packages (in par‐
380 ticular, to avoid trying to represent as changes in the source package
381 uninteresting or perhaps unrepresentable changes to such files) many
382 maintainers arrange for the package clean target to delete these files.
383 dpkg-source does not (with any of the commonly used source formats)
385 represent deletion of binaries (outside debian/) present in upstream.
386 Thus deleting such files in a dpkg-source working tree does not actu‐
387 ally result in them being deleted from the source package. Thus delet‐
388 ing the files in rules clean sweeps this problem under the rug.
389 However, git does always properly record file deletion. Since dgit's
391 principle is that the dgit git tree is the same of dpkg-source -x, that
392 means that a dgit-compatible git tree always contains these files.
393 For the non-maintainer, this can be observed in the following subopti‐
395 mal occurrences:
396 · The package clean target often deletes these files, making the
398 git tree dirty trying to build the source package, etc. This
399 can be fixed by using dgit -wg aka --clean=git, so that the
400 package clean target is never run.
401 · The package build modifies these files, so that builds make the
403 git tree dirty. This can be worked around by using `git reset
404 --hard' after each build (or at least before each commit or
405 push).
406 From the maintainer's point of view, the main consequence is that to
408 make a dgit-compatible git branch it is necessary to commit these files
409 to git. The maintainer has a few additional options for mitigation:
410 for example, it may be possible for the rules file to arrange to do the
411 build in a temporary area, which avoids updating the troublesome files;
412 they can then be left in the git tree without seeing trouble.
413
415 A related problem is other unexpected behaviour by a package's clean
416 target. If a package's rules modify files which are distributed in the
417 package, or simply forget to remove certain files, dgit will complain
418 that the tree is dirty.
419 Again, the solution is to use dgit -wg aka --clean=git, which instructs
421 dgit to use git clean instead of the package's build target, along with
422 perhaps git reset --hard before each build.
423 This is 100% reliable, but has the downside that if you forget to git
425 add or to commit, and then use dgit -wg or git reset --hard, your
426 changes may be lost.
427Debian Project dgit(7)