1dgit(7)                              dgit                              dgit(7)
2
3
4

NAME

6       dgit - principles of operation
7

SUMMARY

9       dgit  treats  the Debian archive as a version control system, and bidi‐
10       rectionally gateways between the archive and git.  The git view of  the
11       package  can  contain  the usual upstream git history, and will be aug‐
12       mented by commits representing uploads done  by  other  developers  not
13       using  dgit.   This git history is stored in a canonical location known
14       as dgit-repos which lives on a dedicated git server.
15
16       git branches suitable for use with dgit can be edited directly in  git,
17       and  used  directly  for  building binary packages.  They can be shared
18       using all conventional means for sharing git branches.  It is not  nec‐
19       essary to use dgit to work with dgitish git branches.  However, dgit is
20       (usually) needed in order to convert to or  from  Debian-format  source
21       packages.
22

SEE ALSO

24       dgit(1)
25              Reference manual and documentation catalogue.
26
27       dgit-*(7)
28              Tutorials and workflow guides.  See dgit(1) for a list.
29

MODEL

31       You  may  use any suitable git workflow with dgit, provided you satisfy
32       dgit's requirements:
33
34       dgit maintains a pseudo-remote called dgit, with one branch per  suite.
35       This remote cannot be used with plain git.
36
37       The  dgit-repos  repository for each package contains one ref per suite
38       named refs/dgit/suite.  These should be pushed to only by  dgit.   They
39       are fast forwarding.  Each push on this branch corresponds to an upload
40       (or attempted upload).
41
42       However, it is perfectly fine to have  other  branches  in  dgit-repos;
43       normally the dgit-repos repo for the package will be accessible via the
44       remote name `origin'.
45
46       dgit push will also  make  signed  tags  called  archive/debian/version
47       (with  version encoded a la DEP-14) and push them to dgit-repos.  These
48       are used at the server to authenticate pushes.
49
50       Uploads made by dgit contain an additional field  Dgit  in  the  source
51       package  .dsc.  (This is added by dgit push.)  This specifies: a commit
52       (an ancestor of the dgit/suite branch) whose tree is identical  to  the
53       unpacked  source upload; the distro to which the upload was made; a tag
54       name which can be used to fetch the git commits; and a url to use as  a
55       hint for the dgit git server for that distro.
56
57       Uploads  not  made  by dgit are represented in git by commits which are
58       synthesised by dgit.  The tree of each such commit corresponds  to  the
59       unpacked  source;  there  is  a commit with the contents, and a pseudo-
60       merge from last known upload -  that  is,  from  the  contents  of  the
61       dgit/suite  branch.   Depending  on the source package format, the con‐
62       tents commit may have a more complex structure, but ultimately it  will
63       be  a  convergence  of stubby branches from origin commits representing
64       the components of the source package.
65
66       dgit expects trees that it works with to have a dgit  (pseudo)  remote.
67       This refers to the dgit-created git view of the corresponding archive.
68
69       The  dgit  archive  tracking view is synthesised locally, on demand, by
70       each copy of dgit.  The tracking view is always  a  descendant  of  the
71       dgit-repos  suite  branch  (if  one  exists), but may be ahead of it if
72       uploads have been done without dgit.   The  archive  tracking  view  is
73       always fast forwarding within each suite.
74
75       dgit  push can operate on any commit which is a descendant of the suite
76       tracking branch.
77
78       dgit does not make a systematic record of  its  imports  of  orig  tar‐
79       ball(s).  So it does not work by finding git tags or branches referring
80       to orig tarball(s).  The orig tarballs are downloaded (by  dgit  clone)
81       into  the  parent  directory,  as with a traditional (non-gitish) dpkg-
82       source workflow.  You need to  retain  these  tarballs  in  the  parent
83       directory  for  dgit  build  and  dgit  push.  (They are not needed for
84       purely-git-based workflows.)
85
86       dgit repositories could be cloned with standard  (git)  methods.   How‐
87       ever,  the dgit repositories do not contain uploads not made with dgit.
88       And for sourceful builds / uploads the orig tarball(s) will need to  be
89       present in the parent directory.
90
91       To  a user looking at the archive, changes pushed in a simple NMU using
92       dgit look like reasonable changes made in an NMU: in  a  `3.0  (quilt)'
93       package the delta from the previous upload is recorded in new patch(es)
94       constructed by dpkg-source.
95

COMBINED SUITES

97       dgit can synthesize a combined view of several underlying suites.  This
98       is requested by specifying, for suite, a comma-separated list:
99
100              mainsuite,subsuite...
101
102       This facility is available with dgit clone, fetch and pull, only.
103
104       dgit  will fetch the same package from each specified underlying suite,
105       separately (as if with dgit fetch).  dgit will then  generate  a  pseu‐
106       domerge commit on the tracking branch remotes/dgit/dgit/suite which has
107       the tip of each of the underlying suites as an ancestor, and which con‐
108       tains  the same as the suite which has the highest version of the pack‐
109       age.
110
111       The package must exist in mainsuite, but need not  exist  in  the  sub‐
112       suites.
113
114       If a specified subsuite starts with - then mainsuite is prepended.
115
116       So, for example, stable,-security means to look for the package in sta‐
117       ble, and stable-security, taking whichever is newer.  If stable is cur‐
118       rently   jessie,   dgit   clone   would   leave   you   on  the  branch
119       dgit/jessie,-security.
120
121       Combined suites are not supported by the dgit build  operations.   This
122       is because those options are intended for building for uploading source
123       packages, and look in the changelog to find  the  relevant  suite.   It
124       does  not  make  sense  to  name a dgit-synthesised combined suite in a
125       changelog, or to try to upload to it.
126
127       When using this facility, it is important to always  specify  the  same
128       suites in the same order: dgit will not make a coherent fast-forwarding
129       history view otherwise.
130
131       The history generated by this feature  is  not  normally  suitable  for
132       merging  back  into  upstreams, as it necessarily contains unattractive
133       pseudomerges.
134

LIMITATIONS

136       Because the synthesis of the suite tracking branches  is  done  locally
137       based  only  on  the current archive state, it will not necessarily see
138       every upload not done with dgit.  Also, different versions of dgit  (or
139       the  software  it  calls)  might  import  the  same  .dscs  differently
140       (although we try to minimise this).  As a consequence, the dgit  track‐
141       ing  views  of the same suite, made by different instances of dgit, may
142       vary.  They will have the same contents, but may  have  different  his‐
143       tory.
144
145       There is no uniform linkage between the tracking branches for different
146       suites.  The Debian infrastructure does not do any automatic import  of
147       uploads  made without dgit.  It would be possible for a distro's infra‐
148       structure to do this; in that case,  different  dgit  client  instances
149       would see exactly the same history.
150
151       There  has been no bulk import of historical uploads into Debian's dgit
152       infrastructure.  To do this it would be necessary to decide whether  to
153       import  existing  vcs  history  (which  might not be faithful to dgit's
154       invariants) or previous non-Dgit uploads (which  would  not  provide  a
155       very rich history).
156
157       git  represents  only file executability.  git does not represent empty
158       directories, or any leaf objects other than plain files  and  symlinks.
159       The  behaviour of Debian source package formats on objects with unusual
160       permissions is complicated.  Some pathological Debian  source  packages
161       will  no  longer  build  if  empty  directories are pruned (or if other
162       things not reproduced by git are  changed).   Such  sources  cannot  be
163       worked with properly in git, and therefore not with dgit either.
164

READ-ONLY DISTROS

166       Distros  which  do  not maintain a set of dgit history git repositories
167       can still be used in a read-only mode with dgit.  Currently  Ubuntu  is
168       configured this way.
169

GITATTRIBUTES

171       git  has  features  which can automatically transform files as they are
172       being copied between  the  working  tree  and  the  git  history.   The
173       attributes  can  be  specified  in  the  source tree itself, in .gitat‐
174       tributes.  See gitattributes(5).
175
176       These  transformations  are  context-sensitive  and  not,  in  general,
177       reversible, so dgit operates on the principle that the dgit git history
178       contains the actual contents of the package.  (When dgit is  manipulat‐
179       ing a .dsc, it does so in a private area, where the transforming gitat‐
180       tributes are defused, to achieve this.)
181
182       If transforming gitattributes are used, they can cause trouble, because
183       the  working  tree  files can differ from the git revision history (and
184       therefore from the source packages).  dgit warns if it finds a  .gitat‐
185       tributes  file  (in  a  package  being fetched or imported), unless the
186       transforming gitattributes have been defused.
187
188       dgit clone and dgit setup-new-tree disable  transforming  gitattributes
189       by default, by creating a suitable .git/info/attributes.  See dgit set‐
190       up-new-tree and dgit setup-gitattributes in dgit(1).
191
192       Note that dgit does not disable gitattributes unless they  would  actu‐
193       ally  interfere with your work on dgit branches.  In particular, gitat‐
194       tributes which affect git archive are not disabled, so .origs you  gen‐
195       erate  by hand can be wrong.  You should consider using git-deborig (1)
196       which gets this right, suppressing the attributes.
197

PACKAGE SOURCE FORMATS

199       If you are not the maintainer, you do  not  need  to  worry  about  the
200       source format of the package.  You can just make changes as you like in
201       git.  If the package is a `3.0 (quilt)' package, the patch  stack  will
202       usually not be represented in the git history.
203

FILE EXECUTABILITY

205       Debian  source  package  formats  do  not  always  faithfully reproduce
206       changes to executability.  But dgit insists that  the  result  of  dgit
207       clone  is  identical  (as  far  as git can represent - see Limitations,
208       above) to the result of dpkg-source -x.
209
210       So files that are executable in your git tree must be executable in the
211       result  of  dpkg-source  -x  (but often aren't).  If a package has such
212       troublesome files, they have to be  non-executable  in  dgit-compatible
213       git branches.
214

FORMAT 3.0 (QUILT)

216       For a format `3.0 (quilt)' source package, dgit may have to make a com‐
217       mit on your current branch to contain metadata used by quilt and  dpkg-
218       source.
219
220       This  is because `3.0 (quilt)' source format represents the patch stack
221       as files in debian/patches/ actually  inside  the  source  tree.   This
222       means  that,  taking  the  whole  tree (as seen by git or ls) (i) dpkg-
223       source cannot represent certain trees, and (ii) packing up  a  tree  in
224       `3.0  (quilt)'  and  then  unpacking  it does not always yield the same
225       tree.
226
227       dgit will automatically work around this  for  you  when  building  and
228       pushing.   The  only thing you need to know is that dgit build, sbuild,
229       etc., may make new commits on your HEAD.  If you're not  a  quilt  user
230       this commit won't contain any changes to files you care about.
231
232       Simply  committing  to source files (whether in debian/ or not, but not
233       to patches) will result in a branch  that  dgit  quilt-fixup  can  lin‐
234       earise.   Other kinds of changes, including editing patches or merging,
235       cannot be handled this way.
236
237       You can explicitly request that dgit do just  this  fixup,  by  running
238       dgit quilt-fixup.
239
240       If  you  are  a  quilt  user you need to know that dgit's git trees are
241       `patches applied packaging branches' and do not contain the .pc  direc‐
242       tory  (which is used by quilt to record which patches are applied).  If
243       you want to manipulate the patch stack you probably want to be  looking
244       at tools like git-debrebase, gbp pq, or git-dpm.
245
246
247   quilt fixup error messages
248       When dgit's quilt fixup fails, it prints messages like this:
249
250       dgit: base trees orig=5531f03d8456b702eab6 o+d/p=135338e9cc253cc85f84
251       dgit: quilt differences: src:  == orig ##     gitignores:  == orig ##
252       dgit: quilt differences:      HEAD ## o+d/p               HEAD ## o+d/p
253       starting quiltify (multiple patches, linear mode)
254
255       dgit: error: quilt fixup cannot be linear.  Stopped at:
256       dgit:  696c9bd5..84ae8f96: changed debian/patches/test-gitignore
257
258
259       orig   is  an import of the .orig tarballs dgit found, with the debian/
260              directory from your  HEAD  substituted.   This  is  a  git  tree
261              object,  not a commit: you can pass its hash to git-diff but not
262              git-log.
263
264
265       o+d/p  is another tree object, which is the same as orig but  with  the
266              patches from debian/patches applied.
267
268
269       HEAD   is of course your own git HEAD.
270
271
272       quilt differences
273              shows  whether  each  of the these trees differs from the others
274              (i) in  upstream  files  excluding  .gitignore  files;  (ii)  in
275              upstream  .gitignore files.  == indicates equality; ## indicates
276              inequality.
277
278       dgit quilt-fixup --quilt=linear walks commits backwards from your  HEAD
279       trying to construct a linear set of additional patches, starting at the
280       end.  It hopes to eventually find an ancestor whose tree  is  identical
281       to o+d/p in all upstream files.
282
283       In the error message, 696c9bd5..84ae8f96 is the first commit child-par‐
284       ent edge which cannot sensibly be either  ignored,  or  turned  into  a
285       patch  in  debian/patches.   In this example, this is because it itself
286       changes files in debian/patches, indicating that something  unusual  is
287       going on and that continuing is not safe.  But you might also see other
288       kinds of troublesome commit or edge.
289
290       Your appropriate response depends on the cause and the context.  If you
291       have  been freely merging your git branch and do not need need a pretty
292       linear patch queue, you can use --quilt=smash (or use the 1.0  or  sin‐
293       gle-debian-patch  source  formats;  see dpkg-source(1).)  If you want a
294       pretty linear series, and this message is unexpected, it can mean  that
295       you  have  unwittingly  committed changes that are not representable by
296       dpkg-source (such as some mode changes).  Or maybe you  just  forgot  a
297       necessary --quilt= option.
298
299       Finally, this problem can occur if you have provided Debian git tooling
300       such as git-debrebase, git-dpm or git-buildpackage  with  upstream  git
301       commit(s)  or  tag(s)  which  are  not 100% identical to your orig tar‐
302       ball(s).
303

SPLIT VIEW AND SPLITTING QUILT MODES

305       When working with git branches intended for use with the `3.0  (quilt)'
306       source format dgit can automatically convert a suitable maintainer-pro‐
307       vided git branch (in one of a variety of formats) into a dgit branch.
308
309       When a splitting quilt mode is selected dgit build  commands  and  dgit
310       push  will,  on  each invocation, convert the user's HEAD into the dgit
311       view, so that it can be built and/or uploaded.
312
313       Split view mode can also be enabled explicitly  with  the  --split-view
314       command line option and the .split-view access configuration key.
315
316       When  split  view  is  in  operation, regardless of the quilt mode, any
317       dgit-generated pseudomerges and any quilt  fixup  commits  will  appear
318       only  in  the dgit view.  dgit push will push the dgit view to the dgit
319       git server.  The dgit view is always a  descendant  of  the  maintainer
320       view.   dgit  push  will  also  make a maintainer view tag according to
321       DEP-14 and push that to the dgit git server.
322
323       Splitting quilt modes must be enabled explicitly (by  the  use  of  the
324       applicable  command line options, subcommands, or configuration).  This
325       is because it is not possible to reliably tell (for example) whether  a
326       git  tree  for a dpkg-source `3.0 (quilt)' package is a patches-applied
327       or patches-unapplied tree.
328
329       Split view conversions are cached in the  ref  dgit-intern/quilt-cache.
330       This should not be manipulated directly.
331

FILES IN THE ORIG TARBALL BUT NOT IN GIT - AUTOTOOLS ETC.

333       This  section is mainly of interest to maintainers who want to use dgit
334       with their existing git history for the Debian package.
335
336       Some developers like to have an extra-clean git tree which lacks  files
337       which  are  normally  found  in source tarballs and therefore in Debian
338       source packages.  For example, it is conventional to  ship  ./configure
339       in the source tarball, but some people prefer not to have it present in
340       the git view of their project.
341
342       dgit requires that the source package unpacks to exactly the same files
343       as  are  in the git commit on which dgit push operates.  So if you just
344       try to dgit push directly from one of these extra-clean  git  branches,
345       it will fail.
346
347       As the maintainer you therefore have the following options:
348
349       ·      Delete  the files from your git branches, and your Debian source
350              packages, and carry the  deletion  as  a  delta  from  upstream.
351              (With  `3.0  (quilt)'  this  means representing the deletions as
352              patches.  You may need to pass --include-removal to  dpkg-source
353              --commit,  or  pass corresponding options to other tools.)  This
354              can make the Debian source package less useful for people  with‐
355              out Debian build infrastructure.
356
357       ·      Persuade  upstream that the source code in their git history and
358              the source they ship as tarballs should be identical.  Of course
359              simply  removing the files from the tarball may make the tarball
360              hard for people to use.
361
362              One answer is to commit the (maybe autogenerated) files, perhaps
363              with  some simple automation to deal with conflicts and spurious
364              changes.  This has the advantage that someone who clones the git
365              repository  finds  the  program just as easy to build as someone
366              who uses the tarball.
367
368       Of course it may also be that the differences are due to  build  system
369       bugs,  which  cause  unintended  files to end up in the source package.
370       dgit will notice this and complain.  You may have  to  fix  these  bugs
371       before you can unify your existing git history with dgit's.
372

FILES IN THE SOURCE PACKAGE BUT NOT IN GIT - DOCS, BINARIES ETC.

374       Some  upstream  tarballs contain build artifacts which upstream expects
375       some users not to want to rebuild (or indeed to find hard to  rebuild),
376       but which in Debian we always rebuild.
377
378       Examples  sometimes include crossbuild firmware binaries and documenta‐
379       tion.  To avoid problems when building updated source packages (in par‐
380       ticular,  to avoid trying to represent as changes in the source package
381       uninteresting or perhaps unrepresentable changes to  such  files)  many
382       maintainers arrange for the package clean target to delete these files.
383
384       dpkg-source  does  not  (with  any of the commonly used source formats)
385       represent deletion of binaries (outside debian/) present  in  upstream.
386       Thus  deleting  such files in a dpkg-source working tree does not actu‐
387       ally result in them being deleted from the source package.  Thus delet‐
388       ing the files in rules clean sweeps this problem under the rug.
389
390       However,  git  does always properly record file deletion.  Since dgit's
391       principle is that the dgit git tree is the same of dpkg-source -x, that
392       means that a dgit-compatible git tree always contains these files.
393
394       For  the non-maintainer, this can be observed in the following subopti‐
395       mal occurrences:
396
397       ·      The package clean target often deletes these files,  making  the
398              git  tree  dirty  trying to build the source package, etc.  This
399              can be fixed by using dgit -wg  aka  --clean=git,  so  that  the
400              package clean target is never run.
401
402       ·      The  package build modifies these files, so that builds make the
403              git tree dirty.  This can be worked around by using  `git  reset
404              --hard'  after  each  build  (or  at least before each commit or
405              push).
406
407       From the maintainer's point of view, the main consequence  is  that  to
408       make a dgit-compatible git branch it is necessary to commit these files
409       to git.  The maintainer has a few additional  options  for  mitigation:
410       for example, it may be possible for the rules file to arrange to do the
411       build in a temporary area, which avoids updating the troublesome files;
412       they can then be left in the git tree without seeing trouble.
413

PROBLEMS WITH PACKAGE CLEAN TARGETS ETC.

415       A  related  problem  is other unexpected behaviour by a package's clean
416       target.  If a package's rules modify files which are distributed in the
417       package,  or  simply forget to remove certain files, dgit will complain
418       that the tree is dirty.
419
420       Again, the solution is to use dgit -wg aka --clean=git, which instructs
421       dgit to use git clean instead of the package's build target, along with
422       perhaps git reset --hard before each build.
423
424       This is 100% reliable, but has the downside that if you forget  to  git
425       add  or  to  commit,  and  then  use dgit -wg or git reset --hard, your
426       changes may be lost.
427
428
429
430Debian Project                                                         dgit(7)
Impressum