1GIT-FAST-IMPORT(1) Git Manual GIT-FAST-IMPORT(1)
2
6 git-fast-import - Backend for fast Git data importers
7
9 frontend | git fast-import [<options>]
10
12 This program is usually not what the end user wants to run directly.
13 Most end users want to use one of the existing frontend programs, which
14 parses a specific type of foreign source and feeds the contents stored
15 there to git fast-import.
16 fast-import reads a mixed command/data stream from standard input and
18 writes one or more packfiles directly into the current repository. When
19 EOF is received on standard input, fast import writes out updated
20 branch and tag refs, fully updating the current repository with the
21 newly imported data.
22 The fast-import backend itself can import into an empty repository (one
24 that has already been initialized by git init) or incrementally update
25 an existing populated repository. Whether or not incremental imports
26 are supported from a particular foreign source depends on the frontend
27 program in use.
28
30 --force
31 Force updating modified existing branches, even if doing so would
32 cause commits to be lost (as the new commit does not contain the
33 old commit).
34 --quiet
36 Disable the output shown by --stats, making fast-import usually be
37 silent when it is successful. However, if the import stream has
38 directives intended to show user output (e.g. progress
39 directives), the corresponding messages will still be shown.
40 --stats
42 Display some basic statistics about the objects fast-import has
43 created, the packfiles they were stored into, and the memory used
44 by fast-import during this run. Showing this output is currently
45 the default, but can be disabled with --quiet.
46 --allow-unsafe-features
48 Many command-line options can be provided as part of the
49 fast-import stream itself by using the feature or option commands.
50 However, some of these options are unsafe (e.g., allowing
51 fast-import to access the filesystem outside of the repository).
52 These options are disabled by default, but can be allowed by
53 providing this option on the command line. This currently impacts
54 only the export-marks, import-marks, and import-marks-if-exists
55 feature commands.
56 Only enable this option if you trust the program generating the
58 fast-import stream! This option is enabled automatically for
59 remote-helpers that use the `import` capability, as they are
60 already trusted to run their own code.
61 Options for Frontends
63 --cat-blob-fd=<fd>
64 Write responses to get-mark, cat-blob, and ls queries to the file
65 descriptor <fd> instead of stdout. Allows progress output intended
66 for the end-user to be separated from other output.
67 --date-format=<fmt>
69 Specify the type of dates the frontend will supply to fast-import
70 within author, committer and tagger commands. See “Date Formats”
71 below for details about which formats are supported, and their
72 syntax.
73 --done
75 Terminate with error if there is no done command at the end of the
76 stream. This option might be useful for detecting errors that cause
77 the frontend to terminate before it has started to write a stream.
78 Locations of Marks Files
80 --export-marks=<file>
81 Dumps the internal marks table to <file> when complete. Marks are
82 written one per line as :markid SHA-1. Frontends can use this file
83 to validate imports after they have been completed, or to save the
84 marks table across incremental runs. As <file> is only opened and
85 truncated at checkpoint (or completion) the same path can also be
86 safely given to --import-marks.
87 --import-marks=<file>
89 Before processing any input, load the marks specified in <file>.
90 The input file must exist, must be readable, and must use the same
91 format as produced by --export-marks. Multiple options may be
92 supplied to import more than one set of marks. If a mark is defined
93 to different values, the last file wins.
94 --import-marks-if-exists=<file>
96 Like --import-marks but instead of erroring out, silently skips the
97 file if it does not exist.
98 --[no-]relative-marks
100 After specifying --relative-marks the paths specified with
101 --import-marks= and --export-marks= are relative to an internal
102 directory in the current repository. In git-fast-import this means
103 that the paths are relative to the .git/info/fast-import directory.
104 However, other importers may use a different location.
105 Relative and non-relative marks may be combined by interweaving
107 --(no-)-relative-marks with the --(import|export)-marks= options.
108 Submodule Rewriting
110 --rewrite-submodules-from=<name>:<file>,
111 --rewrite-submodules-to=<name>:<file>
112 Rewrite the object IDs for the submodule specified by <name> from
113 the values used in the from <file> to those used in the to <file>.
114 The from marks should have been created by git fast-export, and the
115 to marks should have been created by git fast-import when importing
116 that same submodule.
117 <name> may be any arbitrary string not containing a colon
119 character, but the same value must be used with both options when
120 specifying corresponding marks. Multiple submodules may be
121 specified with different values for <name>. It is an error not to
122 use these options in corresponding pairs.
123 These options are primarily useful when converting a repository
125 from one hash algorithm to another; without them, fast-import will
126 fail if it encounters a submodule because it has no way of writing
127 the object ID into the new hash algorithm.
128 Performance and Compression Tuning
130 --active-branches=<n>
131 Maximum number of branches to maintain active at once. See “Memory
132 Utilization” below for details. Default is 5.
133 --big-file-threshold=<n>
135 Maximum size of a blob that fast-import will attempt to create a
136 delta for, expressed in bytes. The default is 512m (512 MiB). Some
137 importers may wish to lower this on systems with constrained
138 memory.
139 --depth=<n>
141 Maximum delta depth, for blob and tree deltification. Default is
142 50.
143 --export-pack-edges=<file>
145 After creating a packfile, print a line of data to <file> listing
146 the filename of the packfile and the last commit on each branch
147 that was written to that packfile. This information may be useful
148 after importing projects whose total object set exceeds the 4 GiB
149 packfile limit, as these commits can be used as edge points during
150 calls to git pack-objects.
151 --max-pack-size=<n>
153 Maximum size of each output packfile. The default is unlimited.
154 fastimport.unpackLimit
156 See git-config(1)
157
159 The design of fast-import allows it to import large projects in a
160 minimum amount of memory usage and processing time. Assuming the
161 frontend is able to keep up with fast-import and feed it a constant
162 stream of data, import times for projects holding 10+ years of history
163 and containing 100,000+ individual commits are generally completed in
164 just 1-2 hours on quite modest (~$2,000 USD) hardware.
165 Most bottlenecks appear to be in foreign source data access (the source
167 just cannot extract revisions fast enough) or disk IO (fast-import
168 writes as fast as the disk will take the data). Imports will run faster
169 if the source data is stored on a different drive than the destination
170 Git repository (due to less IO contention).
171
173 A typical frontend for fast-import tends to weigh in at approximately
174 200 lines of Perl/Python/Ruby code. Most developers have been able to
175 create working importers in just a couple of hours, even though it is
176 their first exposure to fast-import, and sometimes even to Git. This is
177 an ideal situation, given that most conversion tools are throw-away
178 (use once, and never look back).
179
181 Like git push or git fetch, imports handled by fast-import are safe to
182 run alongside parallel git repack -a -d or git gc invocations, or any
183 other Git operation (including git prune, as loose objects are never
184 used by fast-import).
185 fast-import does not lock the branch or tag refs it is actively
187 importing. After the import, during its ref update phase, fast-import
188 tests each existing branch ref to verify the update will be a
189 fast-forward update (the commit stored in the ref is contained in the
190 new history of the commit to be written). If the update is not a
191 fast-forward update, fast-import will skip updating that ref and
192 instead prints a warning message. fast-import will always attempt to
193 update all branch refs, and does not stop on the first failure.
194 Branch updates can be forced with --force, but it’s recommended that
196 this only be used on an otherwise quiet repository. Using --force is
197 not necessary for an initial import into an empty repository.
198
200 fast-import tracks a set of branches in memory. Any branch can be
201 created or modified at any point during the import process by sending a
202 commit command on the input stream. This design allows a frontend
203 program to process an unlimited number of branches simultaneously,
204 generating commits in the order they are available from the source
205 data. It also simplifies the frontend programs considerably.
206 fast-import does not use or alter the current working directory, or any
208 file within it. (It does however update the current Git repository, as
209 referenced by GIT_DIR.) Therefore an import frontend may use the
210 working directory for its own purposes, such as extracting file
211 revisions from the foreign source. This ignorance of the working
212 directory also allows fast-import to run very quickly, as it does not
213 need to perform any costly file update operations when switching
214 between branches.
215
217 With the exception of raw file data (which Git does not interpret) the
218 fast-import input format is text (ASCII) based. This text based format
219 simplifies development and debugging of frontend programs, especially
220 when a higher level language such as Perl, Python or Ruby is being
221 used.
222 fast-import is very strict about its input. Where we say SP below we
224 mean exactly one space. Likewise LF means one (and only one) linefeed
225 and HT one (and only one) horizontal tab. Supplying additional
226 whitespace characters will cause unexpected results, such as branch
227 names or file names with leading or trailing spaces in their name, or
228 early termination of fast-import when it encounters unexpected input.
229 Stream Comments
231 To aid in debugging frontends fast-import ignores any line that begins
232 with # (ASCII pound/hash) up to and including the line ending LF. A
233 comment line may contain any sequence of bytes that does not contain an
234 LF and therefore may be used to include any detailed debugging
235 information that might be specific to the frontend and useful when
236 inspecting a fast-import data stream.
237 Date Formats
239 The following date formats are supported. A frontend should select the
240 format it will use for this import by passing the format name in the
241 --date-format=<fmt> command-line option.
242 raw
244 This is the Git native format and is <time> SP <offutc>. It is also
245 fast-import’s default format, if --date-format was not specified.
246 The time of the event is specified by <time> as the number of
248 seconds since the UNIX epoch (midnight, Jan 1, 1970, UTC) and is
249 written as an ASCII decimal integer.
250 The local offset is specified by <offutc> as a positive or negative
252 offset from UTC. For example EST (which is 5 hours behind UTC)
253 would be expressed in <tz> by “-0500” while UTC is “+0000”. The
254 local offset does not affect <time>; it is used only as an
255 advisement to help formatting routines display the timestamp.
256 If the local offset is not available in the source material, use
258 “+0000”, or the most common local offset. For example many
259 organizations have a CVS repository which has only ever been
260 accessed by users who are located in the same location and time
261 zone. In this case a reasonable offset from UTC could be assumed.
262 Unlike the rfc2822 format, this format is very strict. Any
264 variation in formatting will cause fast-import to reject the value,
265 and some sanity checks on the numeric values may also be performed.
266 raw-permissive
268 This is the same as raw except that no sanity checks on the numeric
269 epoch and local offset are performed. This can be useful when
270 trying to filter or import an existing history with e.g. bogus
271 timezone values.
272 rfc2822
274 This is the standard email format as described by RFC 2822.
275 An example value is “Tue Feb 6 11:22:18 2007 -0500”. The Git parser
277 is accurate, but a little on the lenient side. It is the same
278 parser used by git am when applying patches received from email.
279 Some malformed strings may be accepted as valid dates. In some of
281 these cases Git will still be able to obtain the correct date from
282 the malformed string. There are also some types of malformed
283 strings which Git will parse wrong, and yet consider valid.
284 Seriously malformed strings will be rejected.
285 Unlike the raw format above, the time zone/UTC offset information
287 contained in an RFC 2822 date string is used to adjust the date
288 value to UTC prior to storage. Therefore it is important that this
289 information be as accurate as possible.
290 If the source material uses RFC 2822 style dates, the frontend
292 should let fast-import handle the parsing and conversion (rather
293 than attempting to do it itself) as the Git parser has been well
294 tested in the wild.
295 Frontends should prefer the raw format if the source material
297 already uses UNIX-epoch format, can be coaxed to give dates in that
298 format, or its format is easily convertible to it, as there is no
299 ambiguity in parsing.
300 now
302 Always use the current time and time zone. The literal now must
303 always be supplied for <when>.
304 This is a toy format. The current time and time zone of this system
306 is always copied into the identity string at the time it is being
307 created by fast-import. There is no way to specify a different time
308 or time zone.
309 This particular format is supplied as it’s short to implement and
311 may be useful to a process that wants to create a new commit right
312 now, without needing to use a working directory or git
313 update-index.
314 If separate author and committer commands are used in a commit the
316 timestamps may not match, as the system clock will be polled twice
317 (once for each command). The only way to ensure that both author
318 and committer identity information has the same timestamp is to
319 omit author (thus copying from committer) or to use a date format
320 other than now.
321 Commands
323 fast-import accepts several commands to update the current repository
324 and control the current import process. More detailed discussion (with
325 examples) of each command follows later.
326 commit
328 Creates a new branch or updates an existing branch by creating a
329 new commit and updating the branch to point at the newly created
330 commit.
331 tag
333 Creates an annotated tag object from an existing commit or branch.
334 Lightweight tags are not supported by this command, as they are not
335 recommended for recording meaningful points in time.
336 reset
338 Reset an existing branch (or a new branch) to a specific revision.
339 This command must be used to change a branch to a specific revision
340 without making a commit on it.
341 blob
343 Convert raw file data into a blob, for future use in a commit
344 command. This command is optional and is not needed to perform an
345 import.
346 alias
348 Record that a mark refers to a given object without first creating
349 any new object. Using --import-marks and referring to missing marks
350 will cause fast-import to fail, so aliases can provide a way to set
351 otherwise pruned commits to a valid value (e.g. the nearest
352 non-pruned ancestor).
353 checkpoint
355 Forces fast-import to close the current packfile, generate its
356 unique SHA-1 checksum and index, and start a new packfile. This
357 command is optional and is not needed to perform an import.
358 progress
360 Causes fast-import to echo the entire line to its own standard
361 output. This command is optional and is not needed to perform an
362 import.
363 done
365 Marks the end of the stream. This command is optional unless the
366 done feature was requested using the --done command-line option or
367 feature done command.
368 get-mark
370 Causes fast-import to print the SHA-1 corresponding to a mark to
371 the file descriptor set with --cat-blob-fd, or stdout if
372 unspecified.
373 cat-blob
375 Causes fast-import to print a blob in cat-file --batch format to
376 the file descriptor set with --cat-blob-fd or stdout if
377 unspecified.
378 ls
380 Causes fast-import to print a line describing a directory entry in
381 ls-tree format to the file descriptor set with --cat-blob-fd or
382 stdout if unspecified.
383 feature
385 Enable the specified feature. This requires that fast-import
386 supports the specified feature, and aborts if it does not.
387 option
389 Specify any of the options listed under OPTIONS that do not change
390 stream semantic to suit the frontend’s needs. This command is
391 optional and is not needed to perform an import.
392 commit
394 Create or update a branch with a new commit, recording one logical
395 change to the project.
396 'commit' SP <ref> LF
398 mark?
399 original-oid?
400 ('author' (SP <name>)? SP LT <email> GT SP <when> LF)?
401 'committer' (SP <name>)? SP LT <email> GT SP <when> LF
402 ('encoding' SP <encoding>)?
403 data
404 ('from' SP <commit-ish> LF)?
405 ('merge' SP <commit-ish> LF)*
406 (filemodify | filedelete | filecopy | filerename | filedeleteall | notemodify)*
407 LF?
408 where <ref> is the name of the branch to make the commit on. Typically
410 branch names are prefixed with refs/heads/ in Git, so importing the CVS
411 branch symbol RELENG-1_0 would use refs/heads/RELENG-1_0 for the value
412 of <ref>. The value of <ref> must be a valid refname in Git. As LF is
413 not valid in a Git refname, no quoting or escaping syntax is supported
414 here.
415 A mark command may optionally appear, requesting fast-import to save a
417 reference to the newly created commit for future use by the frontend
418 (see below for format). It is very common for frontends to mark every
419 commit they create, thereby allowing future branch creation from any
420 imported commit.
421 The data command following committer must supply the commit message
423 (see below for data command syntax). To import an empty commit message
424 use a 0 length data. Commit messages are free-form and are not
425 interpreted by Git. Currently they must be encoded in UTF-8, as
426 fast-import does not permit other encodings to be specified.
427 Zero or more filemodify, filedelete, filecopy, filerename,
429 filedeleteall and notemodify commands may be included to update the
430 contents of the branch prior to creating the commit. These commands may
431 be supplied in any order. However it is recommended that a
432 filedeleteall command precede all filemodify, filecopy, filerename and
433 notemodify commands in the same commit, as filedeleteall wipes the
434 branch clean (see below).
435 The LF after the command is optional (it used to be required). Note
437 that for reasons of backward compatibility, if the commit ends with a
438 data command (i.e. it has no from, merge, filemodify, filedelete,
439 filecopy, filerename, filedeleteall or notemodify commands) then two LF
440 commands may appear at the end of the command instead of just one.
441 author
443 An author command may optionally appear, if the author information
444 might differ from the committer information. If author is omitted
445 then fast-import will automatically use the committer’s information
446 for the author portion of the commit. See below for a description
447 of the fields in author, as they are identical to committer.
448 committer
450 The committer command indicates who made this commit, and when they
451 made it.
452 Here <name> is the person’s display name (for example “Com M
454 Itter”) and <email> is the person’s email address
455 (“cm@example.com”). LT and GT are the literal less-than (\x3c) and
456 greater-than (\x3e) symbols. These are required to delimit the
457 email address from the other fields in the line. Note that <name>
458 and <email> are free-form and may contain any sequence of bytes,
459 except LT, GT and LF. <name> is typically UTF-8 encoded.
460 The time of the change is specified by <when> using the date format
462 that was selected by the --date-format=<fmt> command-line option.
463 See “Date Formats” above for the set of supported formats, and
464 their syntax.
465 encoding
467 The optional encoding command indicates the encoding of the commit
468 message. Most commits are UTF-8 and the encoding is omitted, but
469 this allows importing commit messages into git without first
470 reencoding them.
471 from
473 The from command is used to specify the commit to initialize this
474 branch from. This revision will be the first ancestor of the new
475 commit. The state of the tree built at this commit will begin with
476 the state at the from commit, and be altered by the content
477 modifications in this commit.
478 Omitting the from command in the first commit of a new branch will
480 cause fast-import to create that commit with no ancestor. This
481 tends to be desired only for the initial commit of a project. If
482 the frontend creates all files from scratch when making a new
483 branch, a merge command may be used instead of from to start the
484 commit with an empty tree. Omitting the from command on existing
485 branches is usually desired, as the current commit on that branch
486 is automatically assumed to be the first ancestor of the new
487 commit.
488 As LF is not valid in a Git refname or SHA-1 expression, no quoting
490 or escaping syntax is supported within <commit-ish>.
491 Here <commit-ish> is any of the following:
493 • The name of an existing branch already in fast-import’s
495 internal branch table. If fast-import doesn’t know the name,
496 it’s treated as a SHA-1 expression.
497 • A mark reference, :<idnum>, where <idnum> is the mark number.
499 The reason fast-import uses : to denote a mark reference is
501 this character is not legal in a Git branch name. The leading :
502 makes it easy to distinguish between the mark 42 (:42) and the
503 branch 42 (42 or refs/heads/42), or an abbreviated SHA-1 which
504 happened to consist only of base-10 digits.
505 Marks must be declared (via mark) before they can be used.
507 • A complete 40 byte or abbreviated commit SHA-1 in hex.
509 • Any valid Git SHA-1 expression that resolves to a commit. See
511 “SPECIFYING REVISIONS” in gitrevisions(7) for details.
512 • The special null SHA-1 (40 zeros) specifies that the branch is
514 to be removed.
515 The special case of restarting an incremental import from the
517 current branch value should be written as:
518 from refs/heads/branch^0
520 The ^0 suffix is necessary as fast-import does not permit a branch
522 to start from itself, and the branch is created in memory before
523 the from command is even read from the input. Adding ^0 will force
524 fast-import to resolve the commit through Git’s revision parsing
525 library, rather than its internal branch table, thereby loading in
526 the existing value of the branch.
527 merge
529 Includes one additional ancestor commit. The additional ancestry
530 link does not change the way the tree state is built at this
531 commit. If the from command is omitted when creating a new branch,
532 the first merge commit will be the first ancestor of the current
533 commit, and the branch will start out with no files. An unlimited
534 number of merge commands per commit are permitted by fast-import,
535 thereby establishing an n-way merge.
536 Here <commit-ish> is any of the commit specification expressions
538 also accepted by from (see above).
539 filemodify
541 Included in a commit command to add a new file or change the
542 content of an existing file. This command has two different means
543 of specifying the content of the file.
544 External data format
546 The data content for the file was already supplied by a prior
547 blob command. The frontend just needs to connect it.
548 'M' SP <mode> SP <dataref> SP <path> LF
550 Here usually <dataref> must be either a mark reference
552 (:<idnum>) set by a prior blob command, or a full 40-byte SHA-1
553 of an existing Git blob object. If <mode> is 040000` then
554 <dataref> must be the full 40-byte SHA-1 of an existing Git
555 tree object or a mark reference set with --import-marks.
556 Inline data format
558 The data content for the file has not been supplied yet. The
559 frontend wants to supply it as part of this modify command.
560 'M' SP <mode> SP 'inline' SP <path> LF
562 data
563 See below for a detailed description of the data command.
565 In both formats <mode> is the type of file entry, specified in
567 octal. Git only supports the following modes:
568 • 100644 or 644: A normal (not-executable) file. The majority of
570 files in most projects use this mode. If in doubt, this is what
571 you want.
572 • 100755 or 755: A normal, but executable, file.
574 • 120000: A symlink, the content of the file will be the link
576 target.
577 • 160000: A gitlink, SHA-1 of the object refers to a commit in
579 another repository. Git links can only be specified by SHA or
580 through a commit mark. They are used to implement submodules.
581 • 040000: A subdirectory. Subdirectories can only be specified by
583 SHA or through a tree mark set with --import-marks.
584 In both formats <path> is the complete path of the file to be added
586 (if not already existing) or modified (if already existing).
587 A <path> string must use UNIX-style directory separators (forward
589 slash /), may contain any byte other than LF, and must not start
590 with double quote (").
591 A path can use C-style string quoting; this is accepted in all
593 cases and mandatory if the filename starts with double quote or
594 contains LF. In C-style quoting, the complete name should be
595 surrounded with double quotes, and any LF, backslash, or double
596 quote characters must be escaped by preceding them with a backslash
597 (e.g., "path/with\n, \\ and \" in it").
598 The value of <path> must be in canonical form. That is it must not:
600 • contain an empty directory component (e.g. foo//bar is
602 invalid),
603 • end with a directory separator (e.g. foo/ is invalid),
605 • start with a directory separator (e.g. /foo is invalid),
607 • contain the special component . or .. (e.g. foo/./bar and
609 foo/../bar are invalid).
610 The root of the tree can be represented by an empty string as
612 <path>.
613 It is recommended that <path> always be encoded using UTF-8.
615 filedelete
617 Included in a commit command to remove a file or recursively delete
618 an entire directory from the branch. If the file or directory
619 removal makes its parent directory empty, the parent directory will
620 be automatically removed too. This cascades up the tree until the
621 first non-empty directory or the root is reached.
622 'D' SP <path> LF
624 here <path> is the complete path of the file or subdirectory to be
626 removed from the branch. See filemodify above for a detailed
627 description of <path>.
628 filecopy
630 Recursively copies an existing file or subdirectory to a different
631 location within the branch. The existing file or directory must
632 exist. If the destination exists it will be completely replaced by
633 the content copied from the source.
634 'C' SP <path> SP <path> LF
636 here the first <path> is the source location and the second <path>
638 is the destination. See filemodify above for a detailed description
639 of what <path> may look like. To use a source path that contains SP
640 the path must be quoted.
641 A filecopy command takes effect immediately. Once the source
643 location has been copied to the destination any future commands
644 applied to the source location will not impact the destination of
645 the copy.
646 filerename
648 Renames an existing file or subdirectory to a different location
649 within the branch. The existing file or directory must exist. If
650 the destination exists it will be replaced by the source directory.
651 'R' SP <path> SP <path> LF
653 here the first <path> is the source location and the second <path>
655 is the destination. See filemodify above for a detailed description
656 of what <path> may look like. To use a source path that contains SP
657 the path must be quoted.
658 A filerename command takes effect immediately. Once the source
660 location has been renamed to the destination any future commands
661 applied to the source location will create new files there and not
662 impact the destination of the rename.
663 Note that a filerename is the same as a filecopy followed by a
665 filedelete of the source location. There is a slight performance
666 advantage to using filerename, but the advantage is so small that
667 it is never worth trying to convert a delete/add pair in source
668 material into a rename for fast-import. This filerename command is
669 provided just to simplify frontends that already have rename
670 information and don’t want bother with decomposing it into a
671 filecopy followed by a filedelete.
672 filedeleteall
674 Included in a commit command to remove all files (and also all
675 directories) from the branch. This command resets the internal
676 branch structure to have no files in it, allowing the frontend to
677 subsequently add all interesting files from scratch.
678 'deleteall' LF
680 This command is extremely useful if the frontend does not know (or
682 does not care to know) what files are currently on the branch, and
683 therefore cannot generate the proper filedelete commands to update
684 the content.
685 Issuing a filedeleteall followed by the needed filemodify commands
687 to set the correct content will produce the same results as sending
688 only the needed filemodify and filedelete commands. The
689 filedeleteall approach may however require fast-import to use
690 slightly more memory per active branch (less than 1 MiB for even
691 most large projects); so frontends that can easily obtain only the
692 affected paths for a commit are encouraged to do so.
693 notemodify
695 Included in a commit <notes_ref> command to add a new note
696 annotating a <commit-ish> or change this annotation contents.
697 Internally it is similar to filemodify 100644 on <commit-ish> path
698 (maybe split into subdirectories). It’s not advised to use any
699 other commands to write to the <notes_ref> tree except
700 filedeleteall to delete all existing notes in this tree. This
701 command has two different means of specifying the content of the
702 note.
703 External data format
705 The data content for the note was already supplied by a prior
706 blob command. The frontend just needs to connect it to the
707 commit that is to be annotated.
708 'N' SP <dataref> SP <commit-ish> LF
710 Here <dataref> can be either a mark reference (:<idnum>) set by
712 a prior blob command, or a full 40-byte SHA-1 of an existing
713 Git blob object.
714 Inline data format
716 The data content for the note has not been supplied yet. The
717 frontend wants to supply it as part of this modify command.
718 'N' SP 'inline' SP <commit-ish> LF
720 data
721 See below for a detailed description of the data command.
723 In both formats <commit-ish> is any of the commit specification
725 expressions also accepted by from (see above).
726 mark
728 Arranges for fast-import to save a reference to the current object,
729 allowing the frontend to recall this object at a future point in time,
730 without knowing its SHA-1. Here the current object is the object
731 creation command the mark command appears within. This can be commit,
732 tag, and blob, but commit is the most common usage.
733 'mark' SP ':' <idnum> LF
735 where <idnum> is the number assigned by the frontend to this mark. The
737 value of <idnum> is expressed as an ASCII decimal integer. The value 0
738 is reserved and cannot be used as a mark. Only values greater than or
739 equal to 1 may be used as marks.
740 New marks are created automatically. Existing marks can be moved to
742 another object simply by reusing the same <idnum> in another mark
743 command.
744 original-oid
746 Provides the name of the object in the original source control system.
747 fast-import will simply ignore this directive, but filter processes
748 which operate on and modify the stream before feeding to fast-import
749 may have uses for this information
750 'original-oid' SP <object-identifier> LF
752 where <object-identifier> is any string not containing LF.
754 tag
756 Creates an annotated tag referring to a specific commit. To create
757 lightweight (non-annotated) tags see the reset command below.
758 'tag' SP <name> LF
760 mark?
761 'from' SP <commit-ish> LF
762 original-oid?
763 'tagger' (SP <name>)? SP LT <email> GT SP <when> LF
764 data
765 where <name> is the name of the tag to create.
767 Tag names are automatically prefixed with refs/tags/ when stored in
769 Git, so importing the CVS branch symbol RELENG-1_0-FINAL would use just
770 RELENG-1_0-FINAL for <name>, and fast-import will write the
771 corresponding ref as refs/tags/RELENG-1_0-FINAL.
772 The value of <name> must be a valid refname in Git and therefore may
774 contain forward slashes. As LF is not valid in a Git refname, no
775 quoting or escaping syntax is supported here.
776 The from command is the same as in the commit command; see above for
778 details.
779 The tagger command uses the same format as committer within commit;
781 again see above for details.
782 The data command following tagger must supply the annotated tag message
784 (see below for data command syntax). To import an empty tag message use
785 a 0 length data. Tag messages are free-form and are not interpreted by
786 Git. Currently they must be encoded in UTF-8, as fast-import does not
787 permit other encodings to be specified.
788 Signing annotated tags during import from within fast-import is not
790 supported. Trying to include your own PGP/GPG signature is not
791 recommended, as the frontend does not (easily) have access to the
792 complete set of bytes which normally goes into such a signature. If
793 signing is required, create lightweight tags from within fast-import
794 with reset, then create the annotated versions of those tags offline
795 with the standard git tag process.
796 reset
798 Creates (or recreates) the named branch, optionally starting from a
799 specific revision. The reset command allows a frontend to issue a new
800 from command for an existing branch, or to create a new branch from an
801 existing commit without creating a new commit.
802 'reset' SP <ref> LF
804 ('from' SP <commit-ish> LF)?
805 LF?
806 For a detailed description of <ref> and <commit-ish> see above under
808 commit and from.
809 The LF after the command is optional (it used to be required).
811 The reset command can also be used to create lightweight
813 (non-annotated) tags. For example:
814 reset refs/tags/938
816 from :938
817 would create the lightweight tag refs/tags/938 referring to whatever
819 commit mark :938 references.
820 blob
822 Requests writing one file revision to the packfile. The revision is not
823 connected to any commit; this connection must be formed in a subsequent
824 commit command by referencing the blob through an assigned mark.
825 'blob' LF
827 mark?
828 original-oid?
829 data
830 The mark command is optional here as some frontends have chosen to
832 generate the Git SHA-1 for the blob on their own, and feed that
833 directly to commit. This is typically more work than it’s worth
834 however, as marks are inexpensive to store and easy to use.
835 data
837 Supplies raw data (for use as blob/file content, commit messages, or
838 annotated tag messages) to fast-import. Data can be supplied using an
839 exact byte count or delimited with a terminating line. Real frontends
840 intended for production-quality conversions should always use the exact
841 byte count format, as it is more robust and performs better. The
842 delimited format is intended primarily for testing fast-import.
843 Comment lines appearing within the <raw> part of data commands are
845 always taken to be part of the body of the data and are therefore never
846 ignored by fast-import. This makes it safe to import any file/message
847 content whose lines might start with #.
848 Exact byte count format
850 The frontend must specify the number of bytes of data.
851 'data' SP <count> LF
853 <raw> LF?
854 where <count> is the exact number of bytes appearing within <raw>.
856 The value of <count> is expressed as an ASCII decimal integer. The
857 LF on either side of <raw> is not included in <count> and will not
858 be included in the imported data.
859 The LF after <raw> is optional (it used to be required) but
861 recommended. Always including it makes debugging a fast-import
862 stream easier as the next command always starts in column 0 of the
863 next line, even if <raw> did not end with an LF.
864 Delimited format
866 A delimiter string is used to mark the end of the data. fast-import
867 will compute the length by searching for the delimiter. This format
868 is primarily useful for testing and is not recommended for real
869 data.
870 'data' SP '<<' <delim> LF
872 <raw> LF
873 <delim> LF
874 LF?
875 where <delim> is the chosen delimiter string. The string <delim>
877 must not appear on a line by itself within <raw>, as otherwise
878 fast-import will think the data ends earlier than it really does.
879 The LF immediately trailing <raw> is part of <raw>. This is one of
880 the limitations of the delimited format, it is impossible to supply
881 a data chunk which does not have an LF as its last byte.
882 The LF after <delim> LF is optional (it used to be required).
884 alias
886 Record that a mark refers to a given object without first creating any
887 new object.
888 'alias' LF
890 mark
891 'to' SP <commit-ish> LF
892 LF?
893 For a detailed description of <commit-ish> see above under from.
895 checkpoint
897 Forces fast-import to close the current packfile, start a new one, and
898 to save out all current branch refs, tags and marks.
899 'checkpoint' LF
901 LF?
902 Note that fast-import automatically switches packfiles when the current
904 packfile reaches --max-pack-size, or 4 GiB, whichever limit is smaller.
905 During an automatic packfile switch fast-import does not update the
906 branch refs, tags or marks.
907 As a checkpoint can require a significant amount of CPU time and disk
909 IO (to compute the overall pack SHA-1 checksum, generate the
910 corresponding index file, and update the refs) it can easily take
911 several minutes for a single checkpoint command to complete.
912 Frontends may choose to issue checkpoints during extremely large and
914 long running imports, or when they need to allow another Git process
915 access to a branch. However given that a 30 GiB Subversion repository
916 can be loaded into Git through fast-import in about 3 hours, explicit
917 checkpointing may not be necessary.
918 The LF after the command is optional (it used to be required).
920 progress
922 Causes fast-import to print the entire progress line unmodified to its
923 standard output channel (file descriptor 1) when the command is
924 processed from the input stream. The command otherwise has no impact on
925 the current import, or on any of fast-import’s internal state.
926 'progress' SP <any> LF
928 LF?
929 The <any> part of the command may contain any sequence of bytes that
931 does not contain LF. The LF after the command is optional. Callers may
932 wish to process the output through a tool such as sed to remove the
933 leading part of the line, for example:
934 frontend | git fast-import | sed 's/^progress //'
936 Placing a progress command immediately after a checkpoint will inform
938 the reader when the checkpoint has been completed and it can safely
939 access the refs that fast-import updated.
940 get-mark
942 Causes fast-import to print the SHA-1 corresponding to a mark to stdout
943 or to the file descriptor previously arranged with the --cat-blob-fd
944 argument. The command otherwise has no impact on the current import;
945 its purpose is to retrieve SHA-1s that later commits might want to
946 refer to in their commit messages.
947 'get-mark' SP ':' <idnum> LF
949 See “Responses To Commands” below for details about how to read this
951 output safely.
952 cat-blob
954 Causes fast-import to print a blob to a file descriptor previously
955 arranged with the --cat-blob-fd argument. The command otherwise has no
956 impact on the current import; its main purpose is to retrieve blobs
957 that may be in fast-import’s memory but not accessible from the target
958 repository.
959 'cat-blob' SP <dataref> LF
961 The <dataref> can be either a mark reference (:<idnum>) set previously
963 or a full 40-byte SHA-1 of a Git blob, preexisting or ready to be
964 written.
965 Output uses the same format as git cat-file --batch:
967 <sha1> SP 'blob' SP <size> LF
969 <contents> LF
970 This command can be used where a filemodify directive can appear,
972 allowing it to be used in the middle of a commit. For a filemodify
973 using an inline directive, it can also appear right before the data
974 directive.
975 See “Responses To Commands” below for details about how to read this
977 output safely.
978 ls
980 Prints information about the object at a path to a file descriptor
981 previously arranged with the --cat-blob-fd argument. This allows
982 printing a blob from the active commit (with cat-blob) or copying a
983 blob or tree from a previous commit for use in the current one (with
984 filemodify).
985 The ls command can also be used where a filemodify directive can
987 appear, allowing it to be used in the middle of a commit.
988 Reading from the active commit
990 This form can only be used in the middle of a commit. The path
991 names a directory entry within fast-import’s active commit. The
992 path must be quoted in this case.
993 'ls' SP <path> LF
995 Reading from a named tree
997 The <dataref> can be a mark reference (:<idnum>) or the full
998 40-byte SHA-1 of a Git tag, commit, or tree object, preexisting or
999 waiting to be written. The path is relative to the top level of the
1000 tree named by <dataref>.
1001 'ls' SP <dataref> SP <path> LF
1003 See filemodify above for a detailed description of <path>.
1005 Output uses the same format as git ls-tree <tree> -- <path>:
1007 <mode> SP ('blob' | 'tree' | 'commit') SP <dataref> HT <path> LF
1009 The <dataref> represents the blob, tree, or commit object at <path> and
1011 can be used in later get-mark, cat-blob, filemodify, or ls commands.
1012 If there is no file or subtree at that path, git fast-import will
1014 instead report
1015 missing SP <path> LF
1017 See “Responses To Commands” below for details about how to read this
1019 output safely.
1020 feature
1022 Require that fast-import supports the specified feature, or abort if it
1023 does not.
1024 'feature' SP <feature> ('=' <argument>)? LF
1026 The <feature> part of the command may be any one of the following:
1028 date-format, export-marks, relative-marks, no-relative-marks, force
1030 Act as though the corresponding command-line option with a leading
1031 -- was passed on the command line (see OPTIONS, above).
1032 import-marks, import-marks-if-exists
1034 Like --import-marks except in two respects: first, only one
1035 "feature import-marks" or "feature import-marks-if-exists" command
1036 is allowed per stream; second, an --import-marks= or
1037 --import-marks-if-exists command-line option overrides any of these
1038 "feature" commands in the stream; third, "feature
1039 import-marks-if-exists" like a corresponding command-line option
1040 silently skips a nonexistent file.
1041 get-mark, cat-blob, ls
1043 Require that the backend support the get-mark, cat-blob, or ls
1044 command respectively. Versions of fast-import not supporting the
1045 specified command will exit with a message indicating so. This lets
1046 the import error out early with a clear message, rather than
1047 wasting time on the early part of an import before the unsupported
1048 command is detected.
1049 notes
1051 Require that the backend support the notemodify (N) subcommand to
1052 the commit command. Versions of fast-import not supporting notes
1053 will exit with a message indicating so.
1054 done
1056 Error out if the stream ends without a done command. Without this
1057 feature, errors causing the frontend to end abruptly at a
1058 convenient point in the stream can go undetected. This may occur,
1059 for example, if an import front end dies in mid-operation without
1060 emitting SIGTERM or SIGKILL at its subordinate git fast-import
1061 instance.
1062 option
1064 Processes the specified option so that git fast-import behaves in a way
1065 that suits the frontend’s needs. Note that options specified by the
1066 frontend are overridden by any options the user may specify to git
1067 fast-import itself.
1068 'option' SP <option> LF
1070 The <option> part of the command may contain any of the options listed
1072 in the OPTIONS section that do not change import semantics, without the
1073 leading -- and is treated in the same way.
1074 Option commands must be the first commands on the input (not counting
1076 feature commands), to give an option command after any non-option
1077 command is an error.
1078 The following command-line options change import semantics and may
1080 therefore not be passed as option:
1081 • date-format
1083 • import-marks
1085 • export-marks
1087 • cat-blob-fd
1089 • force
1091 done
1093 If the done feature is not in use, treated as if EOF was read. This can
1094 be used to tell fast-import to finish early.
1095 If the --done command-line option or feature done command is in use,
1097 the done command is mandatory and marks the end of the stream.
1098
1100 New objects written by fast-import are not available immediately. Most
1101 fast-import commands have no visible effect until the next checkpoint
1102 (or completion). The frontend can send commands to fill fast-import’s
1103 input pipe without worrying about how quickly they will take effect,
1104 which improves performance by simplifying scheduling.
1105 For some frontends, though, it is useful to be able to read back data
1107 from the current repository as it is being updated (for example when
1108 the source material describes objects in terms of patches to be applied
1109 to previously imported objects). This can be accomplished by connecting
1110 the frontend and fast-import via bidirectional pipes:
1111 mkfifo fast-import-output
1113 frontend <fast-import-output |
1114 git fast-import >fast-import-output
1115 A frontend set up this way can use progress, get-mark, ls, and cat-blob
1117 commands to read information from the import in progress.
1118 To avoid deadlock, such frontends must completely consume any pending
1120 output from progress, ls, get-mark, and cat-blob before performing
1121 writes to fast-import that might block.
1122
1124 If fast-import is supplied invalid input it will terminate with a
1125 non-zero exit status and create a crash report in the top level of the
1126 Git repository it was importing into. Crash reports contain a snapshot
1127 of the internal fast-import state as well as the most recent commands
1128 that lead up to the crash.
1129 All recent commands (including stream comments, file changes and
1131 progress commands) are shown in the command history within the crash
1132 report, but raw file data and commit messages are excluded from the
1133 crash report. This exclusion saves space within the report file and
1134 reduces the amount of buffering that fast-import must perform during
1135 execution.
1136 After writing a crash report fast-import will close the current
1138 packfile and export the marks table. This allows the frontend developer
1139 to inspect the repository state and resume the import from the point
1140 where it crashed. The modified branches and tags are not updated during
1141 a crash, as the import did not complete successfully. Branch and tag
1142 information can be found in the crash report and must be applied
1143 manually if the update is needed.
1144 An example crash:
1146 $ cat >in <<END_OF_INPUT
1148 # my very first test commit
1149 commit refs/heads/master
1150 committer Shawn O. Pearce <spearce> 19283 -0400
1151 # who is that guy anyway?
1152 data <<EOF
1153 this is my commit
1154 EOF
1155 M 644 inline .gitignore
1156 data <<EOF
1157 .gitignore
1158 EOF
1159 M 777 inline bob
1160 END_OF_INPUT
1161 $ git fast-import <in
1163 fatal: Corrupt mode: M 777 inline bob
1164 fast-import: dumping crash report to .git/fast_import_crash_8434
1165 $ cat .git/fast_import_crash_8434
1167 fast-import crash report:
1168 fast-import process: 8434
1169 parent process : 1391
1170 at Sat Sep 1 00:58:12 2007
1171 fatal: Corrupt mode: M 777 inline bob
1173 Most Recent Commands Before Crash
1175 ---------------------------------
1176 # my very first test commit
1177 commit refs/heads/master
1178 committer Shawn O. Pearce <spearce> 19283 -0400
1179 # who is that guy anyway?
1180 data <<EOF
1181 M 644 inline .gitignore
1182 data <<EOF
1183 * M 777 inline bob
1184 Active Branch LRU
1186 -----------------
1187 active_branches = 1 cur, 5 max
1188 pos clock name
1190 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1191 1) 0 refs/heads/master
1192 Inactive Branches
1194 -----------------
1195 refs/heads/master:
1196 status : active loaded dirty
1197 tip commit : 0000000000000000000000000000000000000000
1198 old tree : 0000000000000000000000000000000000000000
1199 cur tree : 0000000000000000000000000000000000000000
1200 commit clock: 0
1201 last pack :
1202 -------------------
1204 END OF CRASH REPORT
1205
1207 The following tips and tricks have been collected from various users of
1208 fast-import, and are offered here as suggestions.
1209 Use One Mark Per Commit
1211 When doing a repository conversion, use a unique mark per commit (mark
1212 :<n>) and supply the --export-marks option on the command line.
1213 fast-import will dump a file which lists every mark and the Git object
1214 SHA-1 that corresponds to it. If the frontend can tie the marks back to
1215 the source repository, it is easy to verify the accuracy and
1216 completeness of the import by comparing each Git commit to the
1217 corresponding source revision.
1218 Coming from a system such as Perforce or Subversion this should be
1220 quite simple, as the fast-import mark can also be the Perforce
1221 changeset number or the Subversion revision number.
1222 Freely Skip Around Branches
1224 Don’t bother trying to optimize the frontend to stick to one branch at
1225 a time during an import. Although doing so might be slightly faster for
1226 fast-import, it tends to increase the complexity of the frontend code
1227 considerably.
1228 The branch LRU builtin to fast-import tends to behave very well, and
1230 the cost of activating an inactive branch is so low that bouncing
1231 around between branches has virtually no impact on import performance.
1232 Handling Renames
1234 When importing a renamed file or directory, simply delete the old
1235 name(s) and modify the new name(s) during the corresponding commit. Git
1236 performs rename detection after-the-fact, rather than explicitly during
1237 a commit.
1238 Use Tag Fixup Branches
1240 Some other SCM systems let the user create a tag from multiple files
1241 which are not from the same commit/changeset. Or to create tags which
1242 are a subset of the files available in the repository.
1243 Importing these tags as-is in Git is impossible without making at least
1245 one commit which “fixes up” the files to match the content of the tag.
1246 Use fast-import’s reset command to reset a dummy branch outside of your
1247 normal branch space to the base commit for the tag, then commit one or
1248 more file fixup commits, and finally tag the dummy branch.
1249 For example since all normal branches are stored under refs/heads/ name
1251 the tag fixup branch TAG_FIXUP. This way it is impossible for the fixup
1252 branch used by the importer to have namespace conflicts with real
1253 branches imported from the source (the name TAG_FIXUP is not
1254 refs/heads/TAG_FIXUP).
1255 When committing fixups, consider using merge to connect the commit(s)
1257 which are supplying file revisions to the fixup branch. Doing so will
1258 allow tools such as git blame to track through the real commit history
1259 and properly annotate the source files.
1260 After fast-import terminates the frontend will need to do rm
1262 .git/TAG_FIXUP to remove the dummy branch.
1263 Import Now, Repack Later
1265 As soon as fast-import completes the Git repository is completely valid
1266 and ready for use. Typically this takes only a very short time, even
1267 for considerably large projects (100,000+ commits).
1268 However repacking the repository is necessary to improve data locality
1270 and access performance. It can also take hours on extremely large
1271 projects (especially if -f and a large --window parameter is used).
1272 Since repacking is safe to run alongside readers and writers, run the
1273 repack in the background and let it finish when it finishes. There is
1274 no reason to wait to explore your new Git project!
1275 If you choose to wait for the repack, don’t try to run benchmarks or
1277 performance tests until repacking is completed. fast-import outputs
1278 suboptimal packfiles that are simply never seen in real use situations.
1279 Repacking Historical Data
1281 If you are repacking very old imported data (e.g. older than the last
1282 year), consider expending some extra CPU time and supplying --window=50
1283 (or higher) when you run git repack. This will take longer, but will
1284 also produce a smaller packfile. You only need to expend the effort
1285 once, and everyone using your project will benefit from the smaller
1286 repository.
1287 Include Some Progress Messages
1289 Every once in a while have your frontend emit a progress message to
1290 fast-import. The contents of the messages are entirely free-form, so
1291 one suggestion would be to output the current month and year each time
1292 the current commit date moves into the next month. Your users will feel
1293 better knowing how much of the data stream has been processed.
1294
1296 When packing a blob fast-import always attempts to deltify against the
1297 last blob written. Unless specifically arranged for by the frontend,
1298 this will probably not be a prior version of the same file, so the
1299 generated delta will not be the smallest possible. The resulting
1300 packfile will be compressed, but will not be optimal.
1301 Frontends which have efficient access to all revisions of a single file
1303 (for example reading an RCS/CVS ,v file) can choose to supply all
1304 revisions of that file as a sequence of consecutive blob commands. This
1305 allows fast-import to deltify the different file revisions against each
1306 other, saving space in the final packfile. Marks can be used to later
1307 identify individual file revisions during a sequence of commit
1308 commands.
1309 The packfile(s) created by fast-import do not encourage good disk
1311 access patterns. This is caused by fast-import writing the data in the
1312 order it is received on standard input, while Git typically organizes
1313 data within packfiles to make the most recent (current tip) data appear
1314 before historical data. Git also clusters commits together, speeding up
1315 revision traversal through better cache locality.
1316 For this reason it is strongly recommended that users repack the
1318 repository with git repack -a -d after fast-import completes, allowing
1319 Git to reorganize the packfiles for faster data access. If blob deltas
1320 are suboptimal (see above) then also adding the -f option to force
1321 recomputation of all deltas can significantly reduce the final packfile
1322 size (30-50% smaller can be quite typical).
1323 Instead of running git repack you can also run git gc --aggressive,
1325 which will also optimize other things after an import (e.g. pack loose
1326 refs). As noted in the "AGGRESSIVE" section in git-gc(1) the
1327 --aggressive option will find new deltas with the -f option to git-
1328 repack(1). For the reasons elaborated on above using --aggressive after
1329 a fast-import is one of the few cases where it’s known to be
1330 worthwhile.
1331
1333 There are a number of factors which affect how much memory fast-import
1334 requires to perform an import. Like critical sections of core Git,
1335 fast-import uses its own memory allocators to amortize any overheads
1336 associated with malloc. In practice fast-import tends to amortize any
1337 malloc overheads to 0, due to its use of large block allocations.
1338 per object
1340 fast-import maintains an in-memory structure for every object written
1341 in this execution. On a 32 bit system the structure is 32 bytes, on a
1342 64 bit system the structure is 40 bytes (due to the larger pointer
1343 sizes). Objects in the table are not deallocated until fast-import
1344 terminates. Importing 2 million objects on a 32 bit system will require
1345 approximately 64 MiB of memory.
1346 The object table is actually a hashtable keyed on the object name (the
1348 unique SHA-1). This storage configuration allows fast-import to reuse
1349 an existing or already written object and avoid writing duplicates to
1350 the output packfile. Duplicate blobs are surprisingly common in an
1351 import, typically due to branch merges in the source.
1352 per mark
1354 Marks are stored in a sparse array, using 1 pointer (4 bytes or 8
1355 bytes, depending on pointer size) per mark. Although the array is
1356 sparse, frontends are still strongly encouraged to use marks between 1
1357 and n, where n is the total number of marks required for this import.
1358 per branch
1360 Branches are classified as active and inactive. The memory usage of the
1361 two classes is significantly different.
1362 Inactive branches are stored in a structure which uses 96 or 120 bytes
1364 (32 bit or 64 bit systems, respectively), plus the length of the branch
1365 name (typically under 200 bytes), per branch. fast-import will easily
1366 handle as many as 10,000 inactive branches in under 2 MiB of memory.
1367 Active branches have the same overhead as inactive branches, but also
1369 contain copies of every tree that has been recently modified on that
1370 branch. If subtree include has not been modified since the branch
1371 became active, its contents will not be loaded into memory, but if
1372 subtree src has been modified by a commit since the branch became
1373 active, then its contents will be loaded in memory.
1374 As active branches store metadata about the files contained on that
1376 branch, their in-memory storage size can grow to a considerable size
1377 (see below).
1378 fast-import automatically moves active branches to inactive status
1380 based on a simple least-recently-used algorithm. The LRU chain is
1381 updated on each commit command. The maximum number of active branches
1382 can be increased or decreased on the command line with
1383 --active-branches=.
1384 per active tree
1386 Trees (aka directories) use just 12 bytes of memory on top of the
1387 memory required for their entries (see “per active file” below). The
1388 cost of a tree is virtually 0, as its overhead amortizes out over the
1389 individual file entries.
1390 per active file entry
1392 Files (and pointers to subtrees) within active trees require 52 or 64
1393 bytes (32/64 bit platforms) per entry. To conserve space, file and tree
1394 names are pooled in a common string table, allowing the filename
1395 “Makefile” to use just 16 bytes (after including the string header
1396 overhead) no matter how many times it occurs within the project.
1397 The active branch LRU, when coupled with the filename string pool and
1399 lazy loading of subtrees, allows fast-import to efficiently import
1400 projects with 2,000+ branches and 45,114+ files in a very limited
1401 memory footprint (less than 2.7 MiB per active branch).
1402
1404 Sending SIGUSR1 to the git fast-import process ends the current
1405 packfile early, simulating a checkpoint command. The impatient operator
1406 can use this facility to peek at the objects and refs from an import in
1407 progress, at the cost of some added running time and worse compression.
1408
1410 Everything below this line in this section is selectively included from
1411 the git-config(1) documentation. The content is the same as what’s
1412 found there:
1413 fastimport.unpackLimit
1415 If the number of objects imported by git-fast-import(1) is below
1416 this limit, then the objects will be unpacked into loose object
1417 files. However if the number of imported objects equals or exceeds
1418 this limit then the pack will be stored as a pack. Storing the pack
1419 from a fast-import can make the import operation complete faster,
1420 especially on slow filesystems. If not set, the value of
1421 transfer.unpackLimit is used instead.
1422
1424 git-fast-export(1)
1425
1427 Part of the git(1) suite
1428Git 2.39.1 2023-01-13 GIT-FAST-IMPORT(1)