1GIT-SECRETS() GIT-SECRETS()
2
6 git-secrets - Prevents you from committing passwords and other sensi‐
7 tive information to a git repository.
8 Contents
10 · Synopsis
11 · Description
13 · Installing git-secrets
15 · *nix (Linux/macOS)
17 · Windows
19 · Homebrew (for macOS users)
21 · Advanced configuration
23 · Before making public a repository
25 · Options
27 · Operation Modes
29 · Options for --install
31 · Options for --scan
33 · Options for --list
35 · Options for --add
37 · Options for --register-aws
39 · Options for --aws-provider
41 · Options for --add-provider
43 · Defining prohibited patterns
45 · Ignoring false positives
47 · Secret providers
49 · Example walkthrough
51 · Skipping validation
53 · About
55
57 git secrets --scan [-r|--recursive] [--cached] [--no-index] [--untracked] [<files>...]
58 git secrets --scan-history
59 git secrets --install [-f|--force] [<target-directory>]
60 git secrets --list [--global]
61 git secrets --add [-a|--allowed] [-l|--literal] [--global] <pattern>
62 git secrets --add-provider [--global] <command> [arguments...]
63 git secrets --register-aws [--global]
64 git secrets --aws-provider [<credentials-file>]
65
67 git-secrets scans commits, commit messages, and --no-ff merges to pre‐
68 vent adding secrets into your git repositories. If a commit, commit
69 message, or any commit in a --no-ff merge history matches one of your
70 configured prohibited regular expression patterns, then the commit is
71 rejected.
72
74 git-secrets must be placed somewhere in your PATH so that it is picked
75 up by git when running git secrets.
76 *nix (Linux/macOS)
78 System Message: WARNING/2 (README.rst:, line 43)
79 Title underline too short.
80 \*nix (Linux/macOS)
82 ~~~~~~~~~~~~~~~~~
83 You can use the install target of the provided Makefile to install git
85 secrets and the man page. You can customize the install path using the
86 PREFIX and MANPREFIX variables.
87 make install
89 Windows
91 Run the provided install.ps1 powershell script. This will copy the
92 needed files to an installation directory (%USERPROFILE%/.git-secrets
93 by default) and add the directory to the current user PATH.
94 PS > ./install.ps1
96 Homebrew (for macOS users)
98 brew install git-secrets
99 WARNING:
101 You're not done yet! You MUST install the git hooks for every repo
102 that you wish to use with git secrets --install.
103 Here's a quick example of how to ensure a git repository is scanned for
105 secrets on each commit:
106 cd /path/to/my/repo
108 git secrets --install
109 git secrets --register-aws
110
112 Add a configuration template if you want to add hooks to all reposito‐
113 ries you initialize or clone in the future.
114 git secrets --register-aws --global
116 Add hooks to all your local repositories.
118 git secrets --install ~/.git-templates/git-secrets
120 git config --global init.templateDir ~/.git-templates/git-secrets
121 Add custom providers to scan for security credentials.
123 git secrets --add-provider -- cat /path/to/secret/file/patterns
125
127 With git-secrets is also possible to scan a repository including all
128 revisions:
129 git secrets --scan-history
131
133 Operation Modes
134 Each of these options must appear first on the command line.
135 --install
137 Installs git hooks for a repository. Once the hooks are
138 installed for a git repository, commits and non-fast-forward
139 merges for that repository will be prevented from committing
140 secrets.
141 --scan Scans one or more files for secrets. When a file contains a
143 secret, the matched text from the file being scanned will be
144 written to stdout and the script will exit with a non-zero sta‐
145 tus. Each matched line will be written with the name of the file
146 that matched, a colon, the line number that matched, a colon,
147 and then the line of text that matched. If no files are pro‐
148 vided, all files returned by git ls-files are scanned.
149 --scan-history
151 Scans repository including all revisions. When a file contains a
152 secret, the matched text from the file being scanned will be
153 written to stdout and the script will exit with a non-zero sta‐
154 tus. Each matched line will be written with the name of the file
155 that matched, a colon, the line number that matched, a colon,
156 and then the line of text that matched.
157 --list Lists the git-secrets configuration for the current repo or in
159 the global git config.
160 --add Adds a prohibited or allowed pattern.
162 --add-provider
164 Registers a secret provider. Secret providers are executables
165 that when invoked output prohibited patterns that git-secrets
166 should treat as prohibited.
167 --register-aws
169 Adds common AWS patterns to the git config and ensures that keys
170 present in ~/.aws/credentials are not found in any commit. The
171 following checks are added:
172 · AWS Access Key IDs via
174 (A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}
175 · AWS Secret Access Key assignments via ":" or "=" surrounded by
177 optional quotes
178 · AWS account ID assignments via ":" or "=" surrounded by
180 optional quotes
181 · Allowed patterns for example AWS keys (AKIAIOSFODNN7EXAMPLE
183 and wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY)
184 · Known credentials from ~/.aws/credentials
186 NOTE:
188 While the patterns registered by this command should catch
189 most instances of AWS credentials, these patterns are not
190 guaranteed to catch them all. git-secrets should be used as
191 an extra means of insurance -- you still need to do your due
192 diligence to ensure that you do not commit credentials to a
193 repository.
194 --aws-provider
196 Secret provider that outputs credentials found in an INI file.
197 You can optionally provide the path to an INI file.
198 Options for --install
200 -f, --force
201 Overwrites existing hooks if present.
202 <target-directory>
204 When provided, installs git hooks to the given directory. The
205 current directory is assumed if <target-directory> is not pro‐
206 vided.
207 If the provided <target-directory> is not in a git repository,
209 the directory will be created and hooks will be placed in <tar‐
210 get-directory>/hooks. This can be useful for creating git tem‐
211 plate directories using with git init --template <target-direc‐
212 tory>.
213 You can run git init on a repository that has already been ini‐
215 tialized. From the git init documentation:
216 From the git documentation: Running git init in an existing
217 repository is safe. It will not overwrite things that are
218 already there. The primary reason for rerunning git init is
219 to pick up newly added templates (or to move the repository
220 to another place if --separate-git-dir is given).
221 The following git hooks are installed:
223 1. pre-commit: Used to check if any of the files changed in the
225 commit use prohibited patterns.
226 2. commit-msg: Used to determine if a commit message contains a
228 prohibited patterns.
229 3. prepare-commit-msg: Used to determine if a merge commit will
231 introduce a history that contains a prohibited pattern at any
232 point. Please note that this hook is only invoked for non
233 fast-forward merges.
234 NOTE:
236 Git only allows a single script to be executed per hook. If
237 the repository contains Debian-style subdirectories like
238 pre-commit.d and commit-msg.d, then the git hooks will be
239 installed into these directories, which assumes that you've
240 configured the corresponding hooks to execute all of the
241 scripts found in these directories. If these git subdirecto‐
242 ries are not present, then the git hooks will be installed to
243 the git repo's .git/hooks directory.
244 Examples
246 Install git hooks to the current directory:
247 cd /path/to/my/repository
249 git secrets --install
250 Install git hooks to a repository other than the current directory:
252 git secrets --install /path/to/my/repository
254 Create a git template that has git-secrets installed, and then copy
256 that template into a git repository:
257 git secrets --install ~/.git-templates/git-secrets
259 git init --template ~/.git-templates/git-secrets
260 Overwrite existing hooks if present:
262 git secrets --install -f
264 Options for --scan
266 -r, --recursive
267 Scans the given files recursively. If a directory is encoun‐
268 tered, the directory will be scanned. If -r is not provided,
269 directories will be ignored.
270 -r cannot be used alongside --cached, --no-index, or
272 --untracked.
273 --cached
275 Searches blobs registered in the index file.
276 --no-index
278 Searches files in the current directory that is not managed by
279 git.
280 --untracked
282 In addition to searching in the tracked files in the working
283 tree, --scan also in untracked files.
284 <files>...
286 The path to one or more files on disk to scan for secrets.
287 If no files are provided, all files returned by git ls-files are
289 scanned.
290 Examples
292 Scan all files in the repo:
293 git secrets --scan
295 Scans a single file for secrets:
297 git secrets --scan /path/to/file
299 Scans a directory recursively for secrets:
301 git secrets --scan -r /path/to/directory
303 Scans multiple files for secrets:
305 git secrets --scan /path/to/file /path/to/other/file
307 You can scan by globbing:
309 git secrets --scan /path/to/directory/*
311 Scan from stdin:
313 echo 'hello!' | git secrets --scan -
315 Options for --list
317 --global
318 Lists only git-secrets configuration in the global git config.
319 Options for --add
321 --global
322 Adds patterns to the global git config
323 -l, --literal
325 Escapes special regular expression characters in the provided
326 pattern so that the pattern is searched for literally.
327 -a, --allowed
329 Mark the pattern as allowed instead of prohibited. Allowed pat‐
330 terns are used to filter our false positives.
331 <pattern>
333 The regex pattern to search.
334 Examples
336 Adds a prohibited pattern to the current repo:
337 git secrets --add '[A-Z0-9]{20}'
339 Adds a prohibited pattern to the global git config:
341 git secrets --add --global '[A-Z0-9]{20}'
343 Adds a string that is scanned for literally (+ is escaped):
345 git secrets --add --literal 'foo+bar'
347 Add an allowed pattern:
349 git secrets --add -a 'allowed pattern'
351 Options for --register-aws
353 --global
354 Adds AWS specific configuration variables to the global git con‐
355 fig.
356 Options for --aws-provider
358 [<credentials-file>]
359 If provided, specifies the custom path to an INI file to scan.
360 If not provided, ~/.aws/credentials is assumed.
361 Options for --add-provider
363 --global
364 Adds the provider to the global git config.
365 <command>
367 Provider command to invoke. When invoked the command is expected
368 to write prohibited patterns separated by new lines to stdout.
369 Any extra arguments provided are passed on to the command.
370 Examples
372 Registers a secret provider with arguments:
373 git secrets --add-provider -- git secrets --aws-provider
375 Cats secrets out of a file:
377 git secrets --add-provider -- cat /path/to/secret/file/patterns
379
381 egrep-compatible regular expressions are used to determine if a commit
382 or commit message contains any prohibited patterns. These regular
383 expressions are defined using the git config command. It is important
384 to note that different systems use different versions of egrep. For
385 example, when running on macOS, you will use a different version of
386 egrep than when running on something like Ubuntu (BSD vs GNU).
387 You can add prohibited regular expression patterns to your git config
389 using git secrets --add <pattern>.
390
392 Sometimes a regular expression might match false positives. For exam‐
393 ple, git commit SHAs look a lot like AWS access keys. You can specify
394 many different regular expression patterns as false positives using the
395 following command:
396 git secrets --add --allowed 'my regex pattern'
398 You can also add regular expressions patterns to filter false positives
400 to a .gitallowed file located in the repository's root directory. Lines
401 starting with # are skipped (comment line) and empty lines are also
402 skipped.
403 First, git-secrets will extract all lines from a file that contain a
405 prohibited match. Included in the matched results will be the full path
406 to the name of the file that was matched, followed by ':', followed by
407 the line number that was matched, followed by the entire line from the
408 file that was matched by a secret pattern. Then, if you've defined
409 allowed regular expressions, git-secrets will check to see if all of
410 the matched lines match at least one of your registered allowed regular
411 expressions. If all of the lines that were flagged as secret are can‐
412 celed out by an allowed match, then the subject text does not contain
413 any secrets. If any of the matched lines are not matched by an allowed
414 regular expression, then git-secrets will fail the commit/merge/mes‐
415 sage.
416 IMPORTANT:
418 Just as it is a bad practice to add prohibited patterns that are too
419 greedy, it is also a bad practice to add allowed patterns that are
420 too forgiving. Be sure to test out your patterns using ad-hoc calls
421 to git secrets --scan $filename to ensure they are working as
422 intended.
423
425 Sometimes you want to check for an exact pattern match against a set of
426 known secrets. For example, you might want to ensure that no creden‐
427 tials present in ~/.aws/credentials ever show up in a commit. In these
428 cases, it's better to leave these secrets in one location rather than
429 spread them out across git repositories in git configs. You can use
430 "secret providers" to fetch these types of credentials. A secret
431 provider is an executable that when invoked outputs prohibited patterns
432 separated by new lines.
433 You can add secret providers using the --add-provider command:
435 git secrets --add-provider -- git secrets --aws-provider
437 Notice the use of --. This ensures that any arguments associated with
439 the provider are passed to the provider each time it is invoked when
440 scanning for secrets.
441
443 Let's take a look at an example. Given the following subject text
444 (stored in /tmp/example):
445 This is a test!
447 password=ex@mplepassword
448 password=******
449 More test...
450 And the following registered patterns:
452 git secrets --add 'password\s*=\s*.+'
454 git secrets --add --allowed --literal 'ex@mplepassword'
455 Running git secrets --scan /tmp/example, the result will result in the
457 following error output:
458 /tmp/example:3:password=******
460 [ERROR] Matched prohibited pattern
462 Possible mitigations:
464 - Mark false positives as allowed using: git config --add secrets.allowed ...
465 - List your configured patterns: git config --get-all secrets.patterns
466 - List your configured allowed patterns: git config --get-all secrets.allowed
467 - Use --no-verify if this is a one-time false positive
468 Breaking this down, the prohibited pattern value of password\s*=\s*.+
470 will match the following lines:
471 /tmp/example:2:password=ex@mplepassword
473 /tmp/example:3:password=******
474 ...But the first match will be filtered out due to the fact that it
476 matches the allowed regular expression of ex@mplepassword. Because
477 there is still a remaining line that did not match, it is considered a
478 secret.
479 Because that matching lines are placed on lines that start with the
481 filename and line number (e.g., /tmp/example:3:...), you can create
482 allowed patterns that take filenames and line numbers into account in
483 the regular expression. For example, you could whitelist an entire file
484 using something like:
485 git secrets --add --allowed '/tmp/example:.*'
487 git secrets --scan /tmp/example && echo $?
488 # Outputs: 0
489 Alternatively, you could allow a specific line number of a file if that
491 line is unlikely to change using something like the following:
492 git secrets --add --allowed '/tmp/example:3:.*'
494 git secrets --scan /tmp/example && echo $?
495 # Outputs: 0
496 Keep this in mind when creating allowed patterns to ensure that your
498 allowed patterns are not inadvertently matched due to the fact that the
499 filename is included in the subject text that allowed patterns are
500 matched against.
501
503 Use the --no-verify option in the event of a false positive match in a
504 commit, merge, or commit message. This will skip the execution of the
505 git hook and allow you to make the commit or merge.
506
508 · Author: Michael Dowling
509 · Issue tracker: This project's source code and issue tracker can be
511 found at https://github.com/awslabs/git-secrets
512 · Special thanks to Adrian Vatchinsky and Ari Juels of Cornell Univer‐
514 sity for providing suggestions and feedback.
515 Copyright 2015 Amazon.com, Inc. or its affiliates. All Rights Reserved.
517 GIT-SECRETS()