1bpkg-repository-types(1) General Commands Manual bpkg-repository-types(1)
2
6 bpkg-repository-types - repository types, structure, and URLs
7
9 This help topic describes the repository types recognized by bpkg,
10 their structure, and the format of their URLs. Currently three types of
11 repositories are supported: archive-based pkg, directory-based dir, and
12 version control-based git.
13 The repository location may specify the repository type as part of the
15 URL scheme component in the type+protocol form. For example:
16 git+https://example.com/foo
18 dir+file:///tmp/repo
19 Note that the explicit specification is only needed when the correct
21 type cannot be guessed from the URL. See bpkg-rep-add(1) for details.
22
24 A pkg repository is archive-based. That is, it contains a collection of
25 various packages/versions as archive files. For more information on the
26 structure of pkg repositories refer to The build2 Package Manager Man‐
27 ual. The pkg repository location can be a local directory path or an
28 http(s):// URL.
29
31 A dir repository is directory-based. That is, it contains a collection
32 of various packages as directories but only a single version per pack‐
33 age can be present in such a repository. The dir repository location
34 can be a local directory path or a file:// URL.
35 A dir repository is expected to contain either the manifest or pack‐
37 ages.manifest file in the root directory of the repository. If it only
38 contains manifest, then it is assumed to be a simple, single-package
39 repository with the manifest file being its package manifest. Other‐
40 wise, the packages.manifest file should list the locations of available
41 packages as described in Package List Manifest for dir Repositories
42 (#manifest-package-list-dir).
43 A dir repository may also contain the repositories.manifest file in the
45 root directory of the repository. This file can be used to describe the
46 repository itself as well as specify its prerequisite and complement
47 repositories. See Repository List Manifest (#manifest-repository-list)
48 for details on the format and semantics of this file.
49
51 A git repository is version control-based. That is, it normally con‐
52 tains multiple versions of the same package (but can also contain sev‐
53 eral, usually related, packages in the same repository).
54 A git repository has the same structure and manifest files as the dir
56 repository. See Package List Manifest for dir Repositories (#manifest-
57 package-list-dir) and Repository List Manifest (#manifest-repository-
58 list) for details on their format and semantics.
59 Theoretically, a git repository may contain as many package versions as
61 there are commits. Practically, however, we are normally only inter‐
62 ested in a small subset of them while fetching and processing the nec‐
63 essary information for all of them could be prohibitively expensive.
64 As a result, by default, only advertised tags in the refs/tags/v* form
65 where the part after v is also a valid standard version (#module-ver‐
66 sion) are considered to be sources of useful package versions. These
67 commits normally correspond to released versions and are called the de‐
68 fault set. Note that only the latest revision of each such version is
69 considered.
70 Instead of the default set, it is possible to provide a custom set of
72 available versions by specifying one or more commit ids and/or refer‐
73 ences and/or reference patterns in the repository URL fragment (see
74 git-ls-remote(1) for details on advertised references). For example:
75 https://example.com/foo.git#v1.2.3
77 https://example.com/foo.git#master
78 https://example.com/foo.git#af234f56
79 https://example.com/foo.git#tags/releases/*
80 https://example.com/foo.git#HEAD,tags/v1.*.*,heads/feature-*
81 Furthermore, it is possible to expand (or narrow down) the default set
83 using the special ## fragment notation. For example:
84 https://example.com/foo.git##HEAD - default set plus HEAD
86 https://example.com/foo.git##heads/* - default set plus branches
87 https://example.com/foo.git##-v1.* - default set minus v1.*
88 A git repository URL fragment is a comma-separated list of reference
90 filters in the following form:
91 [refname][@commit]
93 Either refname, commit, or both must be specified. If both are speci‐
95 fied then refname is only used to minimize the amount of data fetched
96 and commit is expected to belong to its history. For example:
97 .../foo.git#master@48fba3625d65941bb85a39061bcf795d4949c778
99 The refname part can be an abbreviated commit id or an advertised ref‐
101 erence or reference pattern under refs/. While commit must be the com‐
102 plete, 40-characters SHA1 that need not be advertised. For convenience,
103 a 40-characters filter that consists of only hexadecimal digits is as‐
104 sumed to be commit even if not prefixed with @. In an unlikely event
105 this produces an incorrect result, the @-form with omitted commit can
106 be used. For example:
107 .../foo.git#48fba3625d65941bb85a39061bcf795d4949c778 (commit id)
109 .../foo.git#deadbeefdeadbeefdeadbeefdeadbeefdeadbeef@ (reference)
110 The refname part can use the * and ? wildcard pattern characters with
112 the standard semantics as well as the ** character sequence which
113 matches in subdirectories, recursively. For example:
114 .../foo.git#tags/v* - tags/v1.2.3 but not tags/old/v0.1.0
116 .../foo.git#tags/v** - tags/v1.2.3 and tags/old/v0.1.0
117 A relative refname is searched for in refs/, refs/tags/, and
119 refs/heads/ as well as among symbolic references like HEAD. To anchor
120 it to refs/ make it absolute, for example:
121 .../foo.git#tags/v* - refs/tags/v1.2.3 but also refs/heads/tags/voo
123 .../foo.git#/tags/v* - refs/tags/v1.2.3 only
124 While a refname pattern is allowed not match any references, a non-pat‐
126 tern that doesn't resolve to a reference is invalid.
127 If a refname starts with minus (-) then it is treated as an exclusion
129 filter – any references that it matches are excluded from the set in‐
130 cluded by the preceding filters (or the default set). For example:
131 .../foo.git#v*,-v1.* - exclude v1.* from v*
133 .../foo.git##-v1.* - exclude v1.* from default set
134 To support specifying literal leading minus, a refname that starts with
136 plus (+) is treated as an inclusion filter. For example:
137 .../foo.git#+x - include x
139 .../foo.git#+-x - include -x
140 .../foo.git#++x - include +x
141 Currently supported git protocols are git://, ssh:// (but not scp
143 pseudo-URL syntax), http://, and https:// for remote repositories and
144 file:// for local repositories. While bpkg tries to minimize the amount
145 of information (history) fetched, it is not always possible for some
146 protocols and/or server configurations, as discussed next.
147 A git repository accessible via http(s):// can use either dumb or smart
149 protocol (refer to the git documentation for details). The dumb proto‐
150 col provides only limited support for fetch minimization and if this
151 protocol is used, then bpkg has no choice but to download a substantial
152 amount of history.
153 The smart protocol allows fetching of minimal history for tags and
155 branches. Whether this is also possible for (all) commit ids depends
156 on whether the server is configured to allow fetching unadvertised com‐
157 mits. For details, refer to the uploadpack.allowReachableSHA1InWant and
158 uploadpack.allowAnySHA1InWant git configuration values.
159 The git:// and ssh:// protocols are similar to smart http:// in that
161 they support fetching minimal history for tags and branches and may or
162 may not support this for commit ids depending on the server configura‐
163 tion. Note, however, that unlike for http(s)://, for these protocols
164 bpkg does not try to sense if fetching unadvertised commits is allowed
165 and always assumes that it is not. Also note that the sensed or assumed
166 protocol capabilities can be overridden for a git repository URL prefix
167 using the --git-capabilities option (bpkg-common-options(1)).
168 Based on this information, to achieve optimal results the recommended
170 protocol for remote repositories is smart https://. Additionally, if
171 you are planning to refer to unadvertised commit ids, then also con‐
172 sider configuring the server to allow fetching unadvertised commits.
173 The file:// protocol has the same fetch minimization support as git://
175 and is therefore treated the same.
176
178 Send bug reports to the users@build2.org mailing list.
179
181 Copyright (c) 2014-2023 the build2 authors.
182 Permission is granted to copy, distribute and/or modify this document
184 under the terms of the MIT License.
185bpkg 0.16.0 June 2023 bpkg-repository-types(1)