1JQ(1) JQ(1)
2
6 jq - Command-line JSON processor
7
9 jq [options...] filter [files...]
10 jq can transform JSON in various ways, by selecting, iterating, reduc‐
12 ing and otherwise mangling JSON documents. For instance, running the
13 command jq ´map(.price) | add´ will take an array of JSON objects as
14 input and return the sum of their "price" fields.
15 jq can accept text input as well, but by default, jq reads a stream of
17 JSON entities (including numbers and other literals) from stdin. White‐
18 space is only needed to separate entities such as 1 and 2, and true and
19 false. One or more files may be specified, in which case jq will read
20 input from those instead.
21 The options are described in the INVOKING JQ section; they mostly con‐
23 cern input and output formatting. The filter is written in the jq lan‐
24 guage and specifies how to transform the input file or document.
25
27 A jq program is a "filter": it takes an input, and produces an output.
28 There are a lot of builtin filters for extracting a particular field of
29 an object, or converting a number to a string, or various other stan‐
30 dard tasks.
31 Filters can be combined in various ways - you can pipe the output of
33 one filter into another filter, or collect the output of a filter into
34 an array.
35 Some filters produce multiple results, for instance there´s one that
37 produces all the elements of its input array. Piping that filter into a
38 second runs the second filter for each element of the array. Generally,
39 things that would be done with loops and iteration in other languages
40 are just done by gluing filters together in jq.
41 It´s important to remember that every filter has an input and an out‐
43 put. Even literals like "hello" or 42 are filters - they take an input
44 but always produce the same literal as output. Operations that combine
45 two filters, like addition, generally feed the same input to both and
46 combine the results. So, you can implement an averaging filter as add /
47 length - feeding the input array both to the add filter and the length
48 filter and then performing the division.
49 But that´s getting ahead of ourselves. :) Let´s start with something
51 simpler:
52
54 jq filters run on a stream of JSON data. The input to jq is parsed as a
55 sequence of whitespace-separated JSON values which are passed through
56 the provided filter one at a time. The output(s) of the filter are
57 written to standard out, again as a sequence of whitespace-separated
58 JSON data.
59 Note: it is important to mind the shell´s quoting rules. As a general
61 rule it´s best to always quote (with single-quote characters) the jq
62 program, as too many characters with special meaning to jq are also
63 shell meta-characters. For example, jq "foo" will fail on most Unix
64 shells because that will be the same as jq foo, which will generally
65 fail because foo is not defined. When using the Windows command shell
66 (cmd.exe) it´s best to use double quotes around your jq program when
67 given on the command-line (instead of the -f program-file option), but
68 then double-quotes in the jq program need backslash escaping.
69 You can affect how jq reads and writes its input and output using some
71 command-line options:
72 · --version:
74 Output the jq version and exit with zero.
76 · --seq:
78 Use the application/json-seq MIME type scheme for separating JSON
80 texts in jq´s input and output. This means that an ASCII RS (record
81 separator) character is printed before each value on output and an
82 ASCII LF (line feed) is printed after every output. Input JSON
83 texts that fail to parse are ignored (but warned about), discarding
84 all subsequent input until the next RS. This mode also parses the
85 output of jq without the --seq option.
86 · --stream:
88 Parse the input in streaming fashion, outputing arrays of path and
90 leaf values (scalars and empty arrays or empty objects). For exam‐
91 ple, "a" becomes [[],"a"], and [[],"a",["b"]] becomes [[0],[]],
92 [[1],"a"], and [[1,0],"b"].
93 This is useful for processing very large inputs. Use this in con‐
95 junction with filtering and the reduce and foreach syntax to reduce
96 large inputs incrementally.
97 · --slurp/-s:
99 Instead of running the filter for each JSON object in the input,
101 read the entire input stream into a large array and run the filter
102 just once.
103 · --raw-input/-R:
105 Don´t parse the input as JSON. Instead, each line of text is passed
107 to the filter as a string. If combined with --slurp, then the
108 entire input is passed to the filter as a single long string.
109 · --null-input/-n:
111 Don´t read any input at all! Instead, the filter is run once using
113 null as the input. This is useful when using jq as a simple calcu‐
114 lator or to construct JSON data from scratch.
115 · --compact-output / -c:
117 By default, jq pretty-prints JSON output. Using this option will
119 result in more compact output by instead putting each JSON object
120 on a single line.
121 · --tab:
123 Use a tab for each indentation level instead of two spaces.
125 · --indent n:
127 Use the given number of spaces (no more than 8) for indentation.
129 · --color-output / -C and --monochrome-output / -M:
131 By default, jq outputs colored JSON if writing to a terminal. You
133 can force it to produce color even if writing to a pipe or a file
134 using -C, and disable color with -M.
135 Colors can be configured with the JQ_COLORS environment variable
137 (see below).
138 · --ascii-output / -a:
140 jq usually outputs non-ASCII Unicode codepoints as UTF-8, even if
142 the input specified them as escape sequences (like "\u03bc"). Using
143 this option, you can force jq to produce pure ASCII output with
144 every non-ASCII character replaced with the equivalent escape
145 sequence.
146 · --unbuffered
148 Flush the output after each JSON object is printed (useful if
150 you´re piping a slow data source into jq and piping jq´s output
151 elsewhere).
152 · --sort-keys / -S:
154 Output the fields of each object with the keys in sorted order.
156 · --raw-output / -r:
158 With this option, if the filter´s result is a string then it will
160 be written directly to standard output rather than being formatted
161 as a JSON string with quotes. This can be useful for making jq fil‐
162 ters talk to non-JSON-based systems.
163 · --join-output / -j:
165 Like -r but jq won´t print a newline after each output.
167 · -f filename / --from-file filename:
169 Read filter from the file rather than from a command line, like
171 awk´s -f option. You can also use ´#´ to make comments.
172 · -Ldirectory / -L directory:
174 Prepend directory to the search list for modules. If this option is
176 used then no builtin search list is used. See the section on mod‐
177 ules below.
178 · -e / --exit-status:
180 Sets the exit status of jq to 0 if the last output values was nei‐
182 ther false nor null, 1 if the last output value was either false or
183 null, or 4 if no valid result was ever produced. Normally jq exits
184 with 2 if there was any usage problem or system error, 3 if there
185 was a jq program compile error, or 0 if the jq program ran.
186 Another way to set the exit status is with the halt_error builtin
188 function.
189 · --arg name value:
191 This option passes a value to the jq program as a predefined vari‐
193 able. If you run jq with --arg foo bar, then $foo is available in
194 the program and has the value "bar". Note that value will be
195 treated as a string, so --arg foo 123 will bind $foo to "123".
196 Named arguments are also available to the jq program as
198 $ARGS.named.
199 · --argjson name JSON-text:
201 This option passes a JSON-encoded value to the jq program as a pre‐
203 defined variable. If you run jq with --argjson foo 123, then $foo
204 is available in the program and has the value 123.
205 · --slurpfile variable-name filename:
207 This option reads all the JSON texts in the named file and binds an
209 array of the parsed JSON values to the given global variable. If
210 you run jq with --argfile foo bar, then $foo is available in the
211 program and has an array whose elements correspond to the texts in
212 the file named bar.
213 · --argfile variable-name filename:
215 Do not use. Use --slurpfile instead.
217 (This option is like --slurpfile, but when the file has just one
219 text, then that is used, else an array of texts is used as in
220 --slurpfile.)
221 · --args:
223 Remaining arguments are positional string arguments. These are
225 available to the jq program as $ARGS.positional[].
226 · --jsonargs:
228 Remaining arguments are positional JSON text arguments. These are
230 available to the jq program as $ARGS.positional[].
231 · --run-tests [filename]:
233 Runs the tests in the given file or standard input. This must be
235 the last option given and does not honor all preceding options. The
236 input consists of comment lines, empty lines, and program lines
237 followed by one input line, as many lines of output as are expected
238 (one per output), and a terminating empty line. Compilation failure
239 tests start with a line containing only "%%FAIL", then a line con‐
240 taining the program to compile, then a line containing an error
241 message to compare to the actual.
242 Be warned that this option can change backwards-incompatibly.
244
248 Identity: .
249 The absolute simplest filter is . . This is a filter that takes its
250 input and produces it unchanged as output. That is, this is the iden‐
251 tity operator.
252 Since jq by default pretty-prints all output, this trivial program can
254 be a useful way of formatting JSON output from, say, curl.
255 jq ´.´
259 "Hello, world!"
260 => "Hello, world!"
261 Object Identifier-Index: .foo, .foo.bar
265 The simplest useful filter is .foo. When given a JSON object (aka dic‐
266 tionary or hash) as input, it produces the value at the key "foo", or
267 null if there´s none present.
268 A filter of the form .foo.bar is equivalent to .foo|.bar.
270 This syntax only works for simple, identifier-like keys, that is, keys
272 that are all made of alphanumeric characters and underscore, and which
273 do not start with a digit.
274 If the key contains special characters, you need to surround it with
276 double quotes like this: ."foo$", or else .["foo$"].
277 For example .["foo::bar"] and .["foo.bar"] work while .foo::bar does
279 not, and .foo.bar means .["foo"].["bar"].
280 jq ´.foo´
284 {"foo": 42, "bar": "less interesting data"}
285 => 42
286 jq ´.foo´
288 {"notfoo": true, "alsonotfoo": false}
289 => null
290 jq ´.["foo"]´
292 {"foo": 42}
293 => 42
294 Optional Object Identifier-Index: .foo?
298 Just like .foo, but does not output even an error when . is not an
299 array or an object.
300 jq ´.foo?´
304 {"foo": 42, "bar": "less interesting data"}
305 => 42
306 jq ´.foo?´
308 {"notfoo": true, "alsonotfoo": false}
309 => null
310 jq ´.["foo"]?´
312 {"foo": 42}
313 => 42
314 jq ´[.foo?]´
316 [1,2]
317 => []
318 Generic Object Index: .[<string>]
322 You can also look up fields of an object using syntax like .["foo"]
323 (.foo above is a shorthand version of this, but only for identi‐
324 fier-like strings).
325 Array Index: .[2]
327 When the index value is an integer, .[<value>] can index arrays. Arrays
328 are zero-based, so .[2] returns the third element.
329 Negative indices are allowed, with -1 referring to the last element, -2
331 referring to the next to last element, and so on.
332 jq ´.[0]´
336 [{"name":"JSON", "good":true}, {"name":"XML", "good":false}]
337 => {"name":"JSON", "good":true}
338 jq ´.[2]´
340 [{"name":"JSON", "good":true}, {"name":"XML", "good":false}]
341 => null
342 jq ´.[-2]´
344 [1,2,3]
345 => 2
346 Array/String Slice: .[10:15]
350 The .[10:15] syntax can be used to return a subarray of an array or
351 substring of a string. The array returned by .[10:15] will be of length
352 5, containing the elements from index 10 (inclusive) to index 15
353 (exclusive). Either index may be negative (in which case it counts
354 backwards from the end of the array), or omitted (in which case it
355 refers to the start or end of the array).
356 jq ´.[2:4]´
360 ["a","b","c","d","e"]
361 => ["c", "d"]
362 jq ´.[2:4]´
364 "abcdefghi"
365 => "cd"
366 jq ´.[:3]´
368 ["a","b","c","d","e"]
369 => ["a", "b", "c"]
370 jq ´.[-2:]´
372 ["a","b","c","d","e"]
373 => ["d", "e"]
374 Array/Object Value Iterator: .[]
378 If you use the .[index] syntax, but omit the index entirely, it will
379 return all of the elements of an array. Running .[] with the input
380 [1,2,3] will produce the numbers as three separate results, rather than
381 as a single array.
382 You can also use this on an object, and it will return all the values
384 of the object.
385 jq ´.[]´
389 [{"name":"JSON", "good":true}, {"name":"XML", "good":false}]
390 => {"name":"JSON", "good":true}, {"name":"XML", "good":false}
391 jq ´.[]´
393 []
394 =>
395 jq ´.[]´
397 {"a": 1, "b": 1}
398 => 1, 1
399 .[]?
403 Like .[], but no errors will be output if . is not an array or object.
404 Comma: ,
406 If two filters are separated by a comma, then the same input will be
407 fed into both and the two filters´ output value streams will be con‐
408 catenated in order: first, all of the outputs produced by the left
409 expression, and then all of the outputs produced by the right. For
410 instance, filter .foo, .bar, produces both the "foo" fields and "bar"
411 fields as separate outputs.
412 jq ´.foo, .bar´
416 {"foo": 42, "bar": "something else", "baz": true}
417 => 42, "something else"
418 jq ´.user, .projects[]´
420 {"user":"stedolan", "projects": ["jq", "wikiflow"]}
421 => "stedolan", "jq", "wikiflow"
422 jq ´.[4,2]´
424 ["a","b","c","d","e"]
425 => "e", "c"
426 Pipe: |
430 The | operator combines two filters by feeding the output(s) of the one
431 on the left into the input of the one on the right. It´s pretty much
432 the same as the Unix shell´s pipe, if you´re used to that.
433 If the one on the left produces multiple results, the one on the right
435 will be run for each of those results. So, the expression .[] | .foo
436 retrieves the "foo" field of each element of the input array.
437 Note that .a.b.c is the same as .a | .b | .c.
439 Note too that . is the input value at the particular stage in a "pipe‐
441 line", specifically: where the . expression appears. Thus .a | . | .b
442 is the same as .a.b, as the . in the middle refers to whatever value .a
443 produced.
444 jq ´.[] | .name´
448 [{"name":"JSON", "good":true}, {"name":"XML", "good":false}]
449 => "JSON", "XML"
450 Parenthesis
454 Parenthesis work as a grouping operator just as in any typical program‐
455 ming language.
456 jq ´(. + 2) * 5´
460 1
461 => 15
462
466 jq supports the same set of datatypes as JSON - numbers, strings, bool‐
467 eans, arrays, objects (which in JSON-speak are hashes with only string
468 keys), and "null".
469 Booleans, null, strings and numbers are written the same way as in
471 javascript. Just like everything else in jq, these simple values take
472 an input and produce an output - 42 is a valid jq expression that takes
473 an input, ignores it, and returns 42 instead.
474 Array construction: []
476 As in JSON, [] is used to construct arrays, as in [1,2,3]. The elements
477 of the arrays can be any jq expression, including a pipeline. All of
478 the results produced by all of the expressions are collected into one
479 big array. You can use it to construct an array out of a known quantity
480 of values (as in [.foo, .bar, .baz]) or to "collect" all the results of
481 a filter into an array (as in [.items[].name])
482 Once you understand the "," operator, you can look at jq´s array syntax
484 in a different light: the expression [1,2,3] is not using a built-in
485 syntax for comma-separated arrays, but is instead applying the [] oper‐
486 ator (collect results) to the expression 1,2,3 (which produces three
487 different results).
488 If you have a filter X that produces four results, then the expression
490 [X] will produce a single result, an array of four elements.
491 jq ´[.user, .projects[]]´
495 {"user":"stedolan", "projects": ["jq", "wikiflow"]}
496 => ["stedolan", "jq", "wikiflow"]
497 jq ´[ .[] | . * 2]´
499 [1, 2, 3]
500 => [2, 4, 6]
501 Object Construction: {}
505 Like JSON, {} is for constructing objects (aka dictionaries or hashes),
506 as in: {"a": 42, "b": 17}.
507 If the keys are "identifier-like", then the quotes can be left off, as
509 in {a:42, b:17}. Keys generated by expressions need to be parenthe‐
510 sized, e.g., {("a"+"b"):59}.
511 The value can be any expression (although you may need to wrap it in
513 parentheses if it´s a complicated one), which gets applied to the {}
514 expression´s input (remember, all filters have an input and an output).
515 {foo: .bar}
519 will produce the JSON object {"foo": 42} if given the JSON object
523 {"bar":42, "baz":43} as its input. You can use this to select particu‐
524 lar fields of an object: if the input is an object with "user",
525 "title", "id", and "content" fields and you just want "user" and
526 "title", you can write
527 {user: .user, title: .title}
531 Because that is so common, there´s a shortcut syntax for it: {user,
535 title}.
536 If one of the expressions produces multiple results, multiple dictio‐
538 naries will be produced. If the input´s
539 {"user":"stedolan","titles":["JQ Primer", "More JQ"]}
543 then the expression
547 {user, title: .titles[]}
551 will produce two outputs:
555 {"user":"stedolan", "title": "JQ Primer"}
559 {"user":"stedolan", "title": "More JQ"}
560 Putting parentheses around the key means it will be evaluated as an
564 expression. With the same input as above,
565 {(.user): .titles}
569 produces
573 {"stedolan": ["JQ Primer", "More JQ"]}
577 jq ´{user, title: .titles[]}´
579 {"user":"stedolan","titles":["JQ Primer", "More JQ"]}
580 => {"user":"stedolan", "title": "JQ Primer"}, {"user":"stedolan", "title": "More JQ"}
581 jq ´{(.user): .titles}´
583 {"user":"stedolan","titles":["JQ Primer", "More JQ"]}
584 => {"stedolan": ["JQ Primer", "More JQ"]}
585 Recursive Descent: ..
589 Recursively descends ., producing every value. This is the same as the
590 zero-argument recurse builtin (see below). This is intended to resemble
591 the XPath // operator. Note that ..a does not work; use ..|.a instead.
592 In the example below we use ..|.a? to find all the values of object
593 keys "a" in any object found "below" ..
594 This is particularly useful in conjunction with path(EXP) (also see
596 below) and the ? operator.
597 jq ´..|.a?´
601 [[{"a":1}]]
602 => 1
603
607 Some jq operator (for instance, +) do different things depending on the
608 type of their arguments (arrays, numbers, etc.). However, jq never does
609 implicit type conversions. If you try to add a string to an object
610 you´ll get an error message and no result.
611 Addition: +
613 The operator + takes two filters, applies them both to the same input,
614 and adds the results together. What "adding" means depends on the types
615 involved:
616 · Numbers are added by normal arithmetic.
618 · Arrays are added by being concatenated into a larger array.
620 · Strings are added by being joined into a larger string.
622 · Objects are added by merging, that is, inserting all the key-value
624 pairs from both objects into a single combined object. If both
625 objects contain a value for the same key, the object on the right
626 of the + wins. (For recursive merge use the * operator.)
627 null can be added to any value, and returns the other value unchanged.
631 jq ´.a + 1´
635 {"a": 7}
636 => 8
637 jq ´.a + .b´
639 {"a": [1,2], "b": [3,4]}
640 => [1,2,3,4]
641 jq ´.a + null´
643 {"a": 1}
644 => 1
645 jq ´.a + 1´
647 {}
648 => 1
649 jq ´{a: 1} + {b: 2} + {c: 3} + {a: 42}´
651 null
652 => {"a": 42, "b": 2, "c": 3}
653 Subtraction: -
657 As well as normal arithmetic subtraction on numbers, the - operator can
658 be used on arrays to remove all occurrences of the second array´s ele‐
659 ments from the first array.
660 jq ´4 - .a´
664 {"a":3}
665 => 1
666 jq ´. - ["xml", "yaml"]´
668 ["xml", "yaml", "json"]
669 => ["json"]
670 Multiplication, division, modulo: *, /, and %
674 These infix operators behave as expected when given two numbers. Divi‐
675 sion by zero raises an error. x % y computes x modulo y.
676 Multiplying a string by a number produces the concatenation of that
678 string that many times. "x" * 0 produces null.
679 Dividing a string by another splits the first using the second as sepa‐
681 rators.
682 Multiplying two objects will merge them recursively: this works like
684 addition but if both objects contain a value for the same key, and the
685 values are objects, the two are merged with the same strategy.
686 jq ´10 / . * 3´
690 5
691 => 6
692 jq ´. / ", "´
694 "a, b,c,d, e"
695 => ["a","b,c,d","e"]
696 jq ´{"k": {"a": 1, "b": 2}} * {"k": {"a": 0,"c": 3}}´
698 null
699 => {"k": {"a": 0, "b": 2, "c": 3}}
700 jq ´.[] | (1 / .)?´
702 [1,0,-1]
703 => 1, -1
704 length
708 The builtin function length gets the length of various different types
709 of value:
710 · The length of a string is the number of Unicode codepoints it con‐
712 tains (which will be the same as its JSON-encoded length in bytes
713 if it´s pure ASCII).
714 · The length of an array is the number of elements.
716 · The length of an object is the number of key-value pairs.
718 · The length of null is zero.
720 jq ´.[] | length´ [[1,2], "string", {"a":2}, null] => 2, 6, 1, 0
722 utf8bytelength
726 The builtin function utf8bytelength outputs the number of bytes used to
727 encode a string in UTF-8.
728 jq ´utf8bytelength´
732 "\u03bc"
733 => 2
734 keys, keys_unsorted
738 The builtin function keys, when given an object, returns its keys in an
739 array.
740 The keys are sorted "alphabetically", by unicode codepoint order. This
742 is not an order that makes particular sense in any particular language,
743 but you can count on it being the same for any two objects with the
744 same set of keys, regardless of locale settings.
745 When keys is given an array, it returns the valid indices for that
747 array: the integers from 0 to length-1.
748 The keys_unsorted function is just like keys, but if the input is an
750 object then the keys will not be sorted, instead the keys will roughly
751 be in insertion order.
752 jq ´keys´
756 {"abc": 1, "abcd": 2, "Foo": 3}
757 => ["Foo", "abc", "abcd"]
758 jq ´keys´
760 [42,3,35]
761 => [0,1,2]
762 has(key)
766 The builtin function has returns whether the input object has the given
767 key, or the input array has an element at the given index.
768 has($key) has the same effect as checking whether $key is a member of
770 the array returned by keys, although has will be faster.
771 jq ´map(has("foo"))´
775 [{"foo": 42}, {}]
776 => [true, false]
777 jq ´map(has(2))´
779 [[0,1], ["a","b","c"]]
780 => [false, true]
781 in
785 The builtin function in returns whether or not the input key is in the
786 given object, or the input index corresponds to an element in the given
787 array. It is, essentially, an inversed version of has.
788 jq ´.[] | in({"foo": 42})´
792 ["foo", "bar"]
793 => true, false
794 jq ´map(in([0,1]))´
796 [2, 0]
797 => [false, true]
798 map(x), map_values(x)
802 For any filter x, map(x) will run that filter for each element of the
803 input array, and return the outputs in a new array. map(.+1) will
804 increment each element of an array of numbers.
805 Similarly, map_values(x) will run that filter for each element, but it
807 will return an object when an object is passed.
808 map(x) is equivalent to [.[] | x]. In fact, this is how it´s defined.
810 Similarly, map_values(x) is defined as .[] |= x.
811 jq ´map(.+1)´
815 [1,2,3]
816 => [2,3,4]
817 jq ´map_values(.+1)´
819 {"a": 1, "b": 2, "c": 3}
820 => {"a": 2, "b": 3, "c": 4}
821 path(path_expression)
825 Outputs array representations of the given path expression in .. The
826 outputs are arrays of strings (object keys) and/or numbers (array
827 indices).
828 Path expressions are jq expressions like .a, but also .[]. There are
830 two types of path expressions: ones that can match exactly, and ones
831 that cannot. For example, .a.b.c is an exact match path expression,
832 while .a[].b is not.
833 path(exact_path_expression) will produce the array representation of
835 the path expression even if it does not exist in ., if . is null or an
836 array or an object.
837 path(pattern) will produce array representations of the paths matching
839 pattern if the paths exist in ..
840 Note that the path expressions are not different from normal expres‐
842 sions. The expression path(..|select(type=="boolean")) outputs all the
843 paths to boolean values in ., and only those paths.
844 jq ´path(.a[0].b)´
848 null
849 => ["a",0,"b"]
850 jq ´[path(..)]´
852 {"a":[{"b":1}]}
853 => [[],["a"],["a",0],["a",0,"b"]]
854 del(path_expression)
858 The builtin function del removes a key and its corresponding value from
859 an object.
860 jq ´del(.foo)´
864 {"foo": 42, "bar": 9001, "baz": 42}
865 => {"bar": 9001, "baz": 42}
866 jq ´del(.[1, 2])´
868 ["foo", "bar", "baz"]
869 => ["foo"]
870 getpath(PATHS)
874 The builtin function getpath outputs the values in . found at each path
875 in PATHS.
876 jq ´getpath(["a","b"])´
880 null
881 => null
882 jq ´[getpath(["a","b"], ["a","c"])]´
884 {"a":{"b":0, "c":1}}
885 => [0, 1]
886 setpath(PATHS; VALUE)
890 The builtin function setpath sets the PATHS in . to VALUE.
891 jq ´setpath(["a","b"]; 1)´
895 null
896 => {"a": {"b": 1}}
897 jq ´setpath(["a","b"]; 1)´
899 {"a":{"b":0}}
900 => {"a": {"b": 1}}
901 jq ´setpath([0,"a"]; 1)´
903 null
904 => [{"a":1}]
905 delpaths(PATHS)
909 The builtin function delpaths sets the PATHS in .. PATHS must be an
910 array of paths, where each path is an array of strings and numbers.
911 jq ´delpaths([["a","b"]])´
915 {"a":{"b":1},"x":{"y":2}}
916 => {"a":{},"x":{"y":2}}
917 to_entries, from_entries, with_entries
921 These functions convert between an object and an array of key-value
922 pairs. If to_entries is passed an object, then for each k: v entry in
923 the input, the output array includes {"key": k, "value": v}.
924 from_entries does the opposite conversion, and with_entries(foo) is a
926 shorthand for to_entries | map(foo) | from_entries, useful for doing
927 some operation to all keys and values of an object. from_entries
928 accepts key, Key, name, Name, value and Value as keys.
929 jq ´to_entries´
933 {"a": 1, "b": 2}
934 => [{"key":"a", "value":1}, {"key":"b", "value":2}]
935 jq ´from_entries´
937 [{"key":"a", "value":1}, {"key":"b", "value":2}]
938 => {"a": 1, "b": 2}
939 jq ´with_entries(.key |= "KEY_" + .)´
941 {"a": 1, "b": 2}
942 => {"KEY_a": 1, "KEY_b": 2}
943 select(boolean_expression)
947 The function select(foo) produces its input unchanged if foo returns
948 true for that input, and produces no output otherwise.
949 It´s useful for filtering lists: [1,2,3] | map(select(. >= 2)) will
951 give you [2,3].
952 jq ´map(select(. >= 2))´
956 [1,5,3,0,7]
957 => [5,3,7]
958 jq ´.[] | select(.id == "second")´
960 [{"id": "first", "val": 1}, {"id": "second", "val": 2}]
961 => {"id": "second", "val": 2}
962 arrays, objects, iterables, booleans, numbers, normals, finites, strings,
966 nulls, values, scalars
967 These built-ins select only inputs that are arrays, objects, iterables
968 (arrays or objects), booleans, numbers, normal numbers, finite numbers,
969 strings, null, non-null values, and non-iterables, respectively.
970 jq ´.[]|numbers´
974 [[],{},1,"foo",null,true,false]
975 => 1
976 empty
980 empty returns no results. None at all. Not even null.
981 It´s useful on occasion. You´ll know if you need it :)
983 jq ´1, empty, 2´
987 null
988 => 1, 2
989 jq ´[1,2,empty,3]´
991 null
992 => [1,2,3]
993 error(message)
997 Produces an error, just like .a applied to values other than null and
998 objects would, but with the given message as the error´s value. Errors
999 can be caught with try/catch; see below.
1000 halt
1002 Stops the jq program with no further outputs. jq will exit with exit
1003 status 0.
1004 halt_error, halt_error(exit_code)
1006 Stops the jq program with no further outputs. The input will be printed
1007 on stderr as raw output (i.e., strings will not have double quotes)
1008 with no decoration, not even a newline.
1009 The given exit_code (defaulting to 5) will be jq´s exit status.
1011 For example, "Error: somthing went wrong\n"|halt_error(1).
1013 $__loc__
1015 Produces an object with a "file" key and a "line" key, with the file‐
1016 name and line number where $__loc__ occurs, as values.
1017 jq ´try error("\($__loc__)") catch .´
1021 null
1022 => "{\"file\":\"<top-level>\",\"line\":1}"
1023 paths, paths(node_filter), leaf_paths
1027 paths outputs the paths to all the elements in its input (except it
1028 does not output the empty list, representing . itself).
1029 paths(f) outputs the paths to any values for which f is true. That is,
1031 paths(numbers) outputs the paths to all numeric values.
1032 leaf_paths is an alias of paths(scalars); leaf_paths is deprecated and
1034 will be removed in the next major release.
1035 jq ´[paths]´
1039 [1,[[],{"a":2}]]
1040 => [[0],[1],[1,0],[1,1],[1,1,"a"]]
1041 jq ´[paths(scalars)]´
1043 [1,[[],{"a":2}]]
1044 => [[0],[1,1,"a"]]
1045 add
1049 The filter add takes as input an array, and produces as output the ele‐
1050 ments of the array added together. This might mean summed, concatenated
1051 or merged depending on the types of the elements of the input array -
1052 the rules are the same as those for the + operator (described above).
1053 If the input is an empty array, add returns null.
1055 jq ´add´
1059 ["a","b","c"]
1060 => "abc"
1061 jq ´add´
1063 [1, 2, 3]
1064 => 6
1065 jq ´add´
1067 []
1068 => null
1069 any, any(condition), any(generator; condition)
1073 The filter any takes as input an array of boolean values, and produces
1074 true as output if any of the elements of the array are true.
1075 If the input is an empty array, any returns false.
1077 The any(condition) form applies the given condition to the elements of
1079 the input array.
1080 The any(generator; condition) form applies the given condition to all
1082 the outputs of the given generator.
1083 jq ´any´
1087 [true, false]
1088 => true
1089 jq ´any´
1091 [false, false]
1092 => false
1093 jq ´any´
1095 []
1096 => false
1097 all, all(condition), all(generator; condition)
1101 The filter all takes as input an array of boolean values, and produces
1102 true as output if all of the elements of the array are true.
1103 The all(condition) form applies the given condition to the elements of
1105 the input array.
1106 The all(generator; condition) form applies the given condition to all
1108 the outputs of the given generator.
1109 If the input is an empty array, all returns true.
1111 jq ´all´
1115 [true, false]
1116 => false
1117 jq ´all´
1119 [true, true]
1120 => true
1121 jq ´all´
1123 []
1124 => true
1125 flatten, flatten(depth)
1129 The filter flatten takes as input an array of nested arrays, and pro‐
1130 duces a flat array in which all arrays inside the original array have
1131 been recursively replaced by their values. You can pass an argument to
1132 it to specify how many levels of nesting to flatten.
1133 flatten(2) is like flatten, but going only up to two levels deep.
1135 jq ´flatten´
1139 [1, [2], [[3]]]
1140 => [1, 2, 3]
1141 jq ´flatten(1)´
1143 [1, [2], [[3]]]
1144 => [1, 2, [3]]
1145 jq ´flatten´
1147 [[]]
1148 => []
1149 jq ´flatten´
1151 [{"foo": "bar"}, [{"foo": "baz"}]]
1152 => [{"foo": "bar"}, {"foo": "baz"}]
1153 range(upto), range(from;upto) range(from;upto;by)
1157 The range function produces a range of numbers. range(4;10) produces 6
1158 numbers, from 4 (inclusive) to 10 (exclusive). The numbers are produced
1159 as separate outputs. Use [range(4;10)] to get a range as an array.
1160 The one argument form generates numbers from 0 to the given number,
1162 with an increment of 1.
1163 The two argument form generates numbers from from to upto with an
1165 increment of 1.
1166 The three argument form generates numbers from to upto with an incre‐
1168 ment of by.
1169 jq ´range(2;4)´
1173 null
1174 => 2, 3
1175 jq ´[range(2;4)]´
1177 null
1178 => [2,3]
1179 jq ´[range(4)]´
1181 null
1182 => [0,1,2,3]
1183 jq ´[range(0;10;3)]´
1185 null
1186 => [0,3,6,9]
1187 jq ´[range(0;10;-1)]´
1189 null
1190 => []
1191 jq ´[range(0;-5;-1)]´
1193 null
1194 => [0,-1,-2,-3,-4]
1195 floor
1199 The floor function returns the floor of its numeric input.
1200 jq ´floor´
1204 3.14159
1205 => 3
1206 sqrt
1210 The sqrt function returns the square root of its numeric input.
1211 jq ´sqrt´
1215 9
1216 => 3
1217 tonumber
1221 The tonumber function parses its input as a number. It will convert
1222 correctly-formatted strings to their numeric equivalent, leave numbers
1223 alone, and give an error on all other input.
1224 jq ´.[] | tonumber´
1228 [1, "1"]
1229 => 1, 1
1230 tostring
1234 The tostring function prints its input as a string. Strings are left
1235 unchanged, and all other values are JSON-encoded.
1236 jq ´.[] | tostring´
1240 [1, "1", [1]]
1241 => "1", "1", "[1]"
1242 type
1246 The type function returns the type of its argument as a string, which
1247 is one of null, boolean, number, string, array or object.
1248 jq ´map(type)´
1252 [0, false, [], {}, null, "hello"]
1253 => ["number", "boolean", "array", "object", "null", "string"]
1254 infinite, nan, isinfinite, isnan, isfinite, isnormal
1258 Some arithmetic operations can yield infinities and "not a number"
1259 (NaN) values. The isinfinite builtin returns true if its input is infi‐
1260 nite. The isnan builtin returns true if its input is a NaN. The infi‐
1261 nite builtin returns a positive infinite value. The nan builtin returns
1262 a NaN. The isnormal builtin returns true if its input is a normal num‐
1263 ber.
1264 Note that division by zero raises an error.
1266 Currently most arithmetic operations operating on infinities, NaNs, and
1268 sub-normals do not raise errors.
1269 jq ´.[] | (infinite * .) < 0´
1273 [-1, 1]
1274 => true, false
1275 jq ´infinite, nan | type´
1277 null
1278 => "number", "number"
1279 sort, sort_by(path_expression)
1283 The sort functions sorts its input, which must be an array. Values are
1284 sorted in the following order:
1285 · null
1287 · false
1289 · true
1291 · numbers
1293 · strings, in alphabetical order (by unicode codepoint value)
1295 · arrays, in lexical order
1297 · objects
1299 The ordering for objects is a little complex: first they´re compared by
1303 comparing their sets of keys (as arrays in sorted order), and if their
1304 keys are equal then the values are compared key by key.
1305 sort may be used to sort by a particular field of an object, or by
1307 applying any jq filter.
1308 sort_by(foo) compares two elements by comparing the result of foo on
1310 each element.
1311 jq ´sort´
1315 [8,3,null,6]
1316 => [null,3,6,8]
1317 jq ´sort_by(.foo)´
1319 [{"foo":4, "bar":10}, {"foo":3, "bar":100}, {"foo":2, "bar":1}]
1320 => [{"foo":2, "bar":1}, {"foo":3, "bar":100}, {"foo":4, "bar":10}]
1321 group_by(path_expression)
1325 group_by(.foo) takes as input an array, groups the elements having the
1326 same .foo field into separate arrays, and produces all of these arrays
1327 as elements of a larger array, sorted by the value of the .foo field.
1328 Any jq expression, not just a field access, may be used in place of
1330 .foo. The sorting order is the same as described in the sort function
1331 above.
1332 jq ´group_by(.foo)´
1336 [{"foo":1, "bar":10}, {"foo":3, "bar":100}, {"foo":1, "bar":1}]
1337 => [[{"foo":1, "bar":10}, {"foo":1, "bar":1}], [{"foo":3, "bar":100}]]
1338 min, max, min_by(path_exp), max_by(path_exp)
1342 Find the minimum or maximum element of the input array.
1343 The min_by(path_exp) and max_by(path_exp) functions allow you to spec‐
1345 ify a particular field or property to examine, e.g. min_by(.foo) finds
1346 the object with the smallest foo field.
1347 jq ´min´
1351 [5,4,2,7]
1352 => 2
1353 jq ´max_by(.foo)´
1355 [{"foo":1, "bar":14}, {"foo":2, "bar":3}]
1356 => {"foo":2, "bar":3}
1357 unique, unique_by(path_exp)
1361 The unique function takes as input an array and produces an array of
1362 the same elements, in sorted order, with duplicates removed.
1363 The unique_by(path_exp) function will keep only one element for each
1365 value obtained by applying the argument. Think of it as making an array
1366 by taking one element out of every group produced by group.
1367 jq ´unique´
1371 [1,2,5,3,5,3,1,3]
1372 => [1,2,3,5]
1373 jq ´unique_by(.foo)´
1375 [{"foo": 1, "bar": 2}, {"foo": 1, "bar": 3}, {"foo": 4, "bar": 5}]
1376 => [{"foo": 1, "bar": 2}, {"foo": 4, "bar": 5}]
1377 jq ´unique_by(length)´
1379 ["chunky", "bacon", "kitten", "cicada", "asparagus"]
1380 => ["bacon", "chunky", "asparagus"]
1381 reverse
1385 This function reverses an array.
1386 jq ´reverse´
1390 [1,2,3,4]
1391 => [4,3,2,1]
1392 contains(element)
1396 The filter contains(b) will produce true if b is completely contained
1397 within the input. A string B is contained in a string A if B is a sub‐
1398 string of A. An array B is contained in an array A if all elements in B
1399 are contained in any element in A. An object B is contained in object A
1400 if all of the values in B are contained in the value in A with the same
1401 key. All other types are assumed to be contained in each other if they
1402 are equal.
1403 jq ´contains("bar")´
1407 "foobar"
1408 => true
1409 jq ´contains(["baz", "bar"])´
1411 ["foobar", "foobaz", "blarp"]
1412 => true
1413 jq ´contains(["bazzzzz", "bar"])´
1415 ["foobar", "foobaz", "blarp"]
1416 => false
1417 jq ´contains({foo: 12, bar: [{barp: 12}]})´
1419 {"foo": 12, "bar":[1,2,{"barp":12, "blip":13}]}
1420 => true
1421 jq ´contains({foo: 12, bar: [{barp: 15}]})´
1423 {"foo": 12, "bar":[1,2,{"barp":12, "blip":13}]}
1424 => false
1425 indices(s)
1429 Outputs an array containing the indices in . where s occurs. The input
1430 may be an array, in which case if s is an array then the indices output
1431 will be those where all elements in . match those of s.
1432 jq ´indices(", ")´
1436 "a,b, cd, efg, hijk"
1437 => [3,7,12]
1438 jq ´indices(1)´
1440 [0,1,2,1,3,1,4]
1441 => [1,3,5]
1442 jq ´indices([1,2])´
1444 [0,1,2,3,1,4,2,5,1,2,6,7]
1445 => [1,8]
1446 index(s), rindex(s)
1450 Outputs the index of the first (index) or last (rindex) occurrence of s
1451 in the input.
1452 jq ´index(", ")´
1456 "a,b, cd, efg, hijk"
1457 => 3
1458 jq ´rindex(", ")´
1460 "a,b, cd, efg, hijk"
1461 => 12
1462 inside
1466 The filter inside(b) will produce true if the input is completely con‐
1467 tained within b. It is, essentially, an inversed version of contains.
1468 jq ´inside("foobar")´
1472 "bar"
1473 => true
1474 jq ´inside(["foobar", "foobaz", "blarp"])´
1476 ["baz", "bar"]
1477 => true
1478 jq ´inside(["foobar", "foobaz", "blarp"])´
1480 ["bazzzzz", "bar"]
1481 => false
1482 jq ´inside({"foo": 12, "bar":[1,2,{"barp":12, "blip":13}]})´
1484 {"foo": 12, "bar": [{"barp": 12}]}
1485 => true
1486 jq ´inside({"foo": 12, "bar":[1,2,{"barp":12, "blip":13}]})´
1488 {"foo": 12, "bar": [{"barp": 15}]}
1489 => false
1490 startswith(str)
1494 Outputs true if . starts with the given string argument.
1495 jq ´[.[]|startswith("foo")]´
1499 ["fo", "foo", "barfoo", "foobar", "barfoob"]
1500 => [false, true, false, true, false]
1501 endswith(str)
1505 Outputs true if . ends with the given string argument.
1506 jq ´[.[]|endswith("foo")]´
1510 ["foobar", "barfoo"]
1511 => [false, true]
1512 combinations, combinations(n)
1516 Outputs all combinations of the elements of the arrays in the input
1517 array. If given an argument n, it outputs all combinations of n repeti‐
1518 tions of the input array.
1519 jq ´combinations´
1523 [[1,2], [3, 4]]
1524 => [1, 3], [1, 4], [2, 3], [2, 4]
1525 jq ´combinations(2)´
1527 [0, 1]
1528 => [0, 0], [0, 1], [1, 0], [1, 1]
1529 ltrimstr(str)
1533 Outputs its input with the given prefix string removed, if it starts
1534 with it.
1535 jq ´[.[]|ltrimstr("foo")]´
1539 ["fo", "foo", "barfoo", "foobar", "afoo"]
1540 => ["fo","","barfoo","bar","afoo"]
1541 rtrimstr(str)
1545 Outputs its input with the given suffix string removed, if it ends with
1546 it.
1547 jq ´[.[]|rtrimstr("foo")]´
1551 ["fo", "foo", "barfoo", "foobar", "foob"]
1552 => ["fo","","bar","foobar","foob"]
1553 explode
1557 Converts an input string into an array of the string´s codepoint num‐
1558 bers.
1559 jq ´explode´
1563 "foobar"
1564 => [102,111,111,98,97,114]
1565 implode
1569 The inverse of explode.
1570 jq ´implode´
1574 [65, 66, 67]
1575 => "ABC"
1576 split(str)
1580 Splits an input string on the separator argument.
1581 jq ´split(", ")´
1585 "a, b,c,d, e, "
1586 => ["a","b,c,d","e",""]
1587 join(str)
1591 Joins the array of elements given as input, using the argument as sepa‐
1592 rator. It is the inverse of split: that is, running split("foo") |
1593 join("foo") over any input string returns said input string.
1594 Numbers and booleans in the input are converted to strings. Null values
1596 are treated as empty strings. Arrays and objects in the input are not
1597 supported.
1598 jq ´join(", ")´
1602 ["a","b,c,d","e"]
1603 => "a, b,c,d, e"
1604 jq ´join(" ")´
1606 ["a",1,2.3,true,null,false]
1607 => "a 1 2.3 true false"
1608 ascii_downcase, ascii_upcase
1612 Emit a copy of the input string with its alphabetic characters (a-z and
1613 A-Z) converted to the specified case.
1614 while(cond; update)
1616 The while(cond; update) function allows you to repeatedly apply an
1617 update to . until cond is false.
1618 Note that while(cond; update) is internally defined as a recursive jq
1620 function. Recursive calls within while will not consume additional mem‐
1621 ory if update produces at most one output for each input. See advanced
1622 topics below.
1623 jq ´[while(.<100; .*2)]´
1627 1
1628 => [1,2,4,8,16,32,64]
1629 until(cond; next)
1633 The until(cond; next) function allows you to repeatedly apply the
1634 expression next, initially to . then to its own output, until cond is
1635 true. For example, this can be used to implement a factorial function
1636 (see below).
1637 Note that until(cond; next) is internally defined as a recursive jq
1639 function. Recursive calls within until() will not consume additional
1640 memory if next produces at most one output for each input. See advanced
1641 topics below.
1642 jq ´[.,1]|until(.[0] < 1; [.[0] - 1, .[1] * .[0]])|.[1]´
1646 4
1647 => 24
1648 recurse(f), recurse, recurse(f; condition), recurse_down
1652 The recurse(f) function allows you to search through a recursive struc‐
1653 ture, and extract interesting data from all levels. Suppose your input
1654 represents a filesystem:
1655 {"name": "/", "children": [
1659 {"name": "/bin", "children": [
1660 {"name": "/bin/ls", "children": []},
1661 {"name": "/bin/sh", "children": []}]},
1662 {"name": "/home", "children": [
1663 {"name": "/home/stephen", "children": [
1664 {"name": "/home/stephen/jq", "children": []}]}]}]}
1665 Now suppose you want to extract all of the filenames present. You need
1669 to retrieve .name, .children[].name, .children[].children[].name, and
1670 so on. You can do this with:
1671 recurse(.children[]) | .name
1675 When called without an argument, recurse is equivalent to
1679 recurse(.[]?).
1680 recurse(f) is identical to recurse(f; . != null) and can be used with‐
1682 out concerns about recursion depth.
1683 recurse(f; condition) is a generator which begins by emitting . and
1685 then emits in turn .|f, .|f|f, .|f|f|f, ... so long as the computed
1686 value satisfies the condition. For example, to generate all the inte‐
1687 gers, at least in principle, one could write recurse(.+1; true).
1688 For legacy reasons, recurse_down exists as an alias to calling recurse
1690 without arguments. This alias is considered deprecated and will be
1691 removed in the next major release.
1692 The recursive calls in recurse will not consume additional memory when‐
1694 ever f produces at most a single output for each input.
1695 jq ´recurse(.foo[])´
1699 {"foo":[{"foo": []}, {"foo":[{"foo":[]}]}]}
1700 => {"foo":[{"foo":[]},{"foo":[{"foo":[]}]}]}, {"foo":[]}, {"foo":[{"foo":[]}]}, {"foo":[]}
1701 jq ´recurse´
1703 {"a":0,"b":[1]}
1704 => {"a":0,"b":[1]}, 0, [1], 1
1705 jq ´recurse(. * .; . < 20)´
1707 2
1708 => 2, 4, 16
1709 walk(f)
1713 The walk(f) function applies f recursively to every component of the
1714 input entity. When an array is encountered, f is first applied to its
1715 elements and then to the array itself; when an object is encountered, f
1716 is first applied to all the values and then to the object. In practice,
1717 f will usually test the type of its input, as illustrated in the fol‐
1718 lowing examples. The first example highlights the usefulness of pro‐
1719 cessing the elements of an array of arrays before processing the array
1720 itself. The second example shows how all the keys of all the objects
1721 within the input can be considered for alteration.
1722 jq ´walk(if type == "array" then sort else . end)´
1726 [[4, 1, 7], [8, 5, 2], [3, 6, 9]]
1727 => [[1,4,7],[2,5,8],[3,6,9]]
1728 jq ´walk( if type == "object" then with_entries( .key |= sub( "^_+"; "") ) else . end )´
1730 [ { "_a": { "__b": 2 } } ]
1731 => [{"a":{"b":2}}]
1732 $ENV, env
1736 $ENV is an object representing the environment variables as set when
1737 the jq program started.
1738 env outputs an object representing jq´s current environment.
1740 At the moment there is no builtin for setting environment variables.
1742 jq ´$ENV.PAGER´
1746 null
1747 => "less"
1748 jq ´env.PAGER´
1750 null
1751 => "less"
1752 transpose
1756 Transpose a possibly jagged matrix (an array of arrays). Rows are
1757 padded with nulls so the result is always rectangular.
1758 jq ´transpose´
1762 [[1], [2,3]]
1763 => [[1,2],[null,3]]
1764 bsearch(x)
1768 bsearch(x) conducts a binary search for x in the input array. If the
1769 input is sorted and contains x, then bsearch(x) will return its index
1770 in the array; otherwise, if the array is sorted, it will return (-1 -
1771 ix) where ix is an insertion point such that the array would still be
1772 sorted after the insertion of x at ix. If the array is not sorted,
1773 bsearch(x) will return an integer that is probably of no interest.
1774 jq ´bsearch(0)´
1778 [0,1]
1779 => 0
1780 jq ´bsearch(0)´
1782 [1,2,3]
1783 => -1
1784 jq ´bsearch(4) as $ix | if $ix < 0 then .[-(1+$ix)] = 4 else . end´
1786 [1,2,3]
1787 => [1,2,3,4]
1788 String interpolation - \(foo)
1792 Inside a string, you can put an expression inside parens after a back‐
1793 slash. Whatever the expression returns will be interpolated into the
1794 string.
1795 jq ´"The input was \(.), which is one less than \(.+1)"´
1799 42
1800 => "The input was 42, which is one less than 43"
1801 Convert to/from JSON
1805 The tojson and fromjson builtins dump values as JSON texts or parse
1806 JSON texts into values, respectively. The tojson builtin differs from
1807 tostring in that tostring returns strings unmodified, while tojson
1808 encodes strings as JSON strings.
1809 jq ´[.[]|tostring]´
1813 [1, "foo", ["foo"]]
1814 => ["1","foo","[\"foo\"]"]
1815 jq ´[.[]|tojson]´
1817 [1, "foo", ["foo"]]
1818 => ["1","\"foo\"","[\"foo\"]"]
1819 jq ´[.[]|tojson|fromjson]´
1821 [1, "foo", ["foo"]]
1822 => [1,"foo",["foo"]]
1823 Format strings and escaping
1827 The @foo syntax is used to format and escape strings, which is useful
1828 for building URLs, documents in a language like HTML or XML, and so
1829 forth. @foo can be used as a filter on its own, the possible escapings
1830 are:
1831 @text:
1833 Calls tostring, see that function for details.
1835 @json:
1837 Serializes the input as JSON.
1839 @html:
1841 Applies HTML/XML escaping, by mapping the characters <>&´" to
1843 their entity equivalents <, >, &, ', ".
1844 @uri:
1846 Applies percent-encoding, by mapping all reserved URI characters
1848 to a %XX sequence.
1849 @csv:
1851 The input must be an array, and it is rendered as CSV with dou‐
1853 ble quotes for strings, and quotes escaped by repetition.
1854 @tsv:
1856 The input must be an array, and it is rendered as TSV (tab-sepa‐
1858 rated values). Each input array will be printed as a single
1859 line. Fields are separated by a single tab (ascii 0x09). Input
1860 characters line-feed (ascii 0x0a), carriage-return (ascii 0x0d),
1861 tab (ascii 0x09) and backslash (ascii 0x5c) will be output as
1862 escape sequences \n, \r, \t, \\ respectively.
1863 @sh:
1865 The input is escaped suitable for use in a command-line for a
1867 POSIX shell. If the input is an array, the output will be a
1868 series of space-separated strings.
1869 @base64:
1871 The input is converted to base64 as specified by RFC 4648.
1873 @base64d:
1875 The inverse of @base64, input is decoded as specified by RFC
1877 4648. Note: If the decoded string is not UTF-8, the results are
1878 undefined.
1879 This syntax can be combined with string interpolation in a useful way.
1881 You can follow a @foo token with a string literal. The contents of the
1882 string literal will not be escaped. However, all interpolations made
1883 inside that string literal will be escaped. For instance,
1884 @uri "https://www.google.com/search?q=\(.search)"
1888 will produce the following output for the input {"search":"what is
1892 jq?"}:
1893 "https://www.google.com/search?q=what%20is%20jq%3F"
1897 Note that the slashes, question mark, etc. in the URL are not escaped,
1901 as they were part of the string literal.
1902 jq ´@html´
1906 "This works if x < y"
1907 => "This works if x < y"
1908 jq ´@sh "echo \(.)"´
1910 "O´Hara´s Ale"
1911 => "echo ´O´\\´´Hara´\\´´s Ale´"
1912 jq ´@base64´
1914 "This is a message"
1915 => "VGhpcyBpcyBhIG1lc3NhZ2U="
1916 jq ´@base64d´
1918 "VGhpcyBpcyBhIG1lc3NhZ2U="
1919 => "This is a message"
1920 Dates
1924 jq provides some basic date handling functionality, with some
1925 high-level and low-level builtins. In all cases these builtins deal
1926 exclusively with time in UTC.
1927 The fromdateiso8601 builtin parses datetimes in the ISO 8601 format to
1929 a number of seconds since the Unix epoch (1970-01-01T00:00:00Z). The
1930 todateiso8601 builtin does the inverse.
1931 The fromdate builtin parses datetime strings. Currently fromdate only
1933 supports ISO 8601 datetime strings, but in the future it will attempt
1934 to parse datetime strings in more formats.
1935 The todate builtin is an alias for todateiso8601.
1937 The now builtin outputs the current time, in seconds since the Unix
1939 epoch.
1940 Low-level jq interfaces to the C-library time functions are also pro‐
1942 vided: strptime, strftime, strflocaltime, mktime, gmtime, and local‐
1943 time. Refer to your host operating system´s documentation for the for‐
1944 mat strings used by strptime and strftime. Note: these are not neces‐
1945 sarily stable interfaces in jq, particularly as to their localization
1946 functionality.
1947 The gmtime builtin consumes a number of seconds since the Unix epoch
1949 and outputs a "broken down time" representation of Greenwhich Meridian
1950 time as an array of numbers representing (in this order): the year, the
1951 month (zero-based), the day of the month (one-based), the hour of the
1952 day, the minute of the hour, the second of the minute, the day of the
1953 week, and the day of the year -- all one-based unless otherwise stated.
1954 The day of the week number may be wrong on some systems for dates
1955 before March 1st 1900, or after December 31 2099.
1956 The localtime builtin works like the gmtime builtin, but using the
1958 local timezone setting.
1959 The mktime builtin consumes "broken down time" representations of time
1961 output by gmtime and strptime.
1962 The strptime(fmt) builtin parses input strings matching the fmt argu‐
1964 ment. The output is in the "broken down time" representation consumed
1965 by gmtime and output by mktime.
1966 The strftime(fmt) builtin formats a time (GMT) with the given format.
1968 The strflocaltime does the same, but using the local timezone setting.
1969 The format strings for strptime and strftime are described in typical C
1971 library documentation. The format string for ISO 8601 datetime is
1972 "%Y-%m-%dT%H:%M:%SZ".
1973 jq may not support some or all of this date functionality on some sys‐
1975 tems. In particular, the %u and %j specifiers for strptime(fmt) are not
1976 supported on macOS.
1977 jq ´fromdate´
1981 "2015-03-05T23:51:47Z"
1982 => 1425599507
1983 jq ´strptime("%Y-%m-%dT%H:%M:%SZ")´
1985 "2015-03-05T23:51:47Z"
1986 => [2015,2,5,23,51,47,4,63]
1987 jq ´strptime("%Y-%m-%dT%H:%M:%SZ")|mktime´
1989 "2015-03-05T23:51:47Z"
1990 => 1425599507
1991 SQL-Style Operators
1995 jq provides a few SQL-style operators.
1996 INDEX(stream; index_expression):
1998 This builtin produces an object whose keys are computed by the
2000 given index expression applied to each value from the given
2001 stream.
2002 JOIN($idx; stream; idx_expr; join_expr):
2004 This builtin joins the values from the given stream to the given
2006 index. The index´s keys are computed by applying the given index
2007 expression to each value from the given stream. An array of the
2008 value in the stream and the corresponding value from the index
2009 is fed to the given join expression to produce each result.
2010 JOIN($idx; stream; idx_expr):
2012 Same as JOIN($idx; stream; idx_expr; .).
2014 JOIN($idx; idx_expr):
2016 This builtin joins the input . to the given index, applying the
2018 given index expression to . to compute the index key. The join
2019 operation is as described above.
2020 IN(s):
2022 This builtin outputs true if . appears in the given stream, oth‐
2024 erwise it outputs false.
2025 IN(source; s):
2027 This builtin outputs true if any value in the source stream
2029 appears in the second stream, otherwise it outputs false.
2030 builtins
2032 Returns a list of all builtin functions in the format name/arity. Since
2033 functions with the same name but different arities are considered sepa‐
2034 rate functions, all/0, all/1, and all/2 would all be present in the
2035 list.
2036
2038 ==, !=
2039 The expression ´a == b´ will produce ´true´ if the result of a and b
2040 are equal (that is, if they represent equivalent JSON documents) and
2041 ´false´ otherwise. In particular, strings are never considered equal to
2042 numbers. If you´re coming from Javascript, jq´s == is like Javascript´s
2043 === - considering values equal only when they have the same type as
2044 well as the same value.
2045 != is "not equal", and ´a != b´ returns the opposite value of ´a == b´
2047 jq ´.[] == 1´
2051 [1, 1.0, "1", "banana"]
2052 => true, true, false, false
2053 if-then-else
2057 if A then B else C end will act the same as B if A produces a value
2058 other than false or null, but act the same as C otherwise.
2059 Checking for false or null is a simpler notion of "truthiness" than is
2061 found in Javascript or Python, but it means that you´ll sometimes have
2062 to be more explicit about the condition you want: you can´t test
2063 whether, e.g. a string is empty using if .name then A else B end,
2064 you´ll need something more like if (.name | length) > 0 then A else B
2065 end instead.
2066 If the condition A produces multiple results, then B is evaluated once
2068 for each result that is not false or null, and C is evaluated once for
2069 each false or null.
2070 More cases can be added to an if using elif A then B syntax.
2072 jq ´if . == 0 then
2076 "zero" elif . == 1 then "one" else "many" end´ 2 => "many"
2080 >, >=, <=, <
2082 The comparison operators >, >=, <=, < return whether their left argu‐
2083 ment is greater than, greater than or equal to, less than or equal to
2084 or less than their right argument (respectively).
2085 The ordering is the same as that described for sort, above.
2087 jq ´. < 5´
2091 2
2092 => true
2093 and/or/not
2097 jq supports the normal Boolean operators and/or/not. They have the same
2098 standard of truth as if expressions - false and null are considered
2099 "false values", and anything else is a "true value".
2100 If an operand of one of these operators produces multiple results, the
2102 operator itself will produce a result for each input.
2103 not is in fact a builtin function rather than an operator, so it is
2105 called as a filter to which things can be piped rather than with spe‐
2106 cial syntax, as in .foo and .bar | not.
2107 These three only produce the values "true" and "false", and so are only
2109 useful for genuine Boolean operations, rather than the common
2110 Perl/Python/Ruby idiom of "value_that_may_be_null or default". If you
2111 want to use this form of "or", picking between two values rather than
2112 evaluating a condition, see the "//" operator below.
2113 jq ´42 and "a string"´
2117 null
2118 => true
2119 jq ´(true, false) or false´
2121 null
2122 => true, false
2123 jq ´(true, true) and (true, false)´
2125 null
2126 => true, false, true, false
2127 jq ´[true, false | not]´
2129 null
2130 => [false, true]
2131 Alternative operator: //
2135 A filter of the form a // b produces the same results as a, if a pro‐
2136 duces results other than false and null. Otherwise, a // b produces the
2137 same results as b.
2138 This is useful for providing defaults: .foo // 1 will evaluate to 1 if
2140 there´s no .foo element in the input. It´s similar to how or is some‐
2141 times used in Python (jq´s or operator is reserved for strictly Boolean
2142 operations).
2143 jq ´.foo // 42´
2147 {"foo": 19}
2148 => 19
2149 jq ´.foo // 42´
2151 {}
2152 => 42
2153 try-catch
2157 Errors can be caught by using try EXP catch EXP. The first expression
2158 is executed, and if it fails then the second is executed with the error
2159 message. The output of the handler, if any, is output as if it had been
2160 the output of the expression to try.
2161 The try EXP form uses empty as the exception handler.
2163 jq ´try .a catch ". is not an object"´
2167 true
2168 => ". is not an object"
2169 jq ´[.[]|try .a]´
2171 [{}, true, {"a":1}]
2172 => [null, 1]
2173 jq ´try error("some exception") catch .´
2175 true
2176 => "some exception"
2177 Breaking out of control structures
2181 A convenient use of try/catch is to break out of control structures
2182 like reduce, foreach, while, and so on.
2183 For example:
2185 # Repeat an expression until it raises "break" as an
2189 # error, then stop repeating without re-raising the error.
2190 # But if the error caught is not "break" then re-raise it.
2191 try repeat(exp) catch .=="break" then empty else error;
2192 jq has a syntax for named lexical labels to "break" or "go (back) to":
2196 label $out | ... break $out ...
2200 The break $label_name expression will cause the program to to act as
2204 though the nearest (to the left) label $label_name produced empty.
2205 The relationship between the break and corresponding label is lexical:
2207 the label has to be "visible" from the break.
2208 To break out of a reduce, for example:
2210 label $out | reduce .[] as $item (null; if .==false then break $out else ... end)
2214 The following jq program produces a syntax error:
2218 break $out
2222 because no label $out is visible.
2226 Error Suppression / Optional Operator: ?
2228 The ? operator, used as EXP?, is shorthand for try EXP.
2229 jq ´[.[]|(.a)?]´
2233 [{}, true, {"a":1}]
2234 => [null, 1]
2235
2239 jq uses the Oniguruma regular expression library, as do php, ruby,
2240 TextMate, Sublime Text, etc, so the description here will focus on jq
2241 specifics.
2242 The jq regex filters are defined so that they can be used using one of
2244 these patterns:
2245 STRING | FILTER( REGEX )
2249 STRING | FILTER( REGEX; FLAGS )
2250 STRING | FILTER( [REGEX] )
2251 STRING | FILTER( [REGEX, FLAGS] )
2252 where: * STRING, REGEX and FLAGS are jq strings and subject to jq
2256 string interpolation; * REGEX, after string interpolation, should be a
2257 valid PCRE regex; * FILTER is one of test, match, or capture, as
2258 described below.
2259 FLAGS is a string consisting of one of more of the supported flags:
2261 · g - Global search (find all matches, not just the first)
2263 · i - Case insensitive search
2265 · m - Multi line mode (´.´ will match newlines)
2267 · n - Ignore empty matches
2269 · p - Both s and m modes are enabled
2271 · s - Single line mode (´^´ -> ´\A´, ´$´ -> ´\Z´)
2273 · l - Find longest possible matches
2275 · x - Extended regex format (ignore whitespace and comments)
2277 To match whitespace in an x pattern use an escape such as \s, e.g.
2281 · test( "a\sb", "x" ).
2283 Note that certain flags may also be specified within REGEX, e.g.
2287 · jq -n ´("test", "TEst", "teST", "TEST") | test( "(?i)te(?-i)st" )´
2289 evaluates to: true, true, false, false.
2293 test(val), test(regex; flags)
2295 Like match, but does not return match objects, only true or false for
2296 whether or not the regex matches the input.
2297 jq ´test("foo")´
2301 "foo"
2302 => true
2303 jq ´.[] | test("a b c # spaces are ignored"; "ix")´
2305 ["xabcd", "ABC"]
2306 => true, true
2307 match(val), match(regex; flags)
2311 match outputs an object for each match it finds. Matches have the fol‐
2312 lowing fields:
2313 · offset - offset in UTF-8 codepoints from the beginning of the input
2315 · length - length in UTF-8 codepoints of the match
2317 · string - the string that it matched
2319 · captures - an array of objects representing capturing groups.
2321 Capturing group objects have the following fields:
2325 · offset - offset in UTF-8 codepoints from the beginning of the input
2327 · length - length in UTF-8 codepoints of this capturing group
2329 · string - the string that was captured
2331 · name - the name of the capturing group (or null if it was unnamed)
2333 Capturing groups that did not match anything return an offset of -1
2337 jq ´match("(abc)+"; "g")´
2341 "abc abc"
2342 => {"offset": 0, "length": 3, "string": "abc", "captures": [{"offset": 0, "length": 3, "string": "abc", "name": null}]}, {"offset": 4, "length": 3, "string": "abc", "captures": [{"offset": 4, "length": 3, "string": "abc", "name": null}]}
2343 jq ´match("foo")´
2345 "foo bar foo"
2346 => {"offset": 0, "length": 3, "string": "foo", "captures": []}
2347 jq ´match(["foo", "ig"])´
2349 "foo bar FOO"
2350 => {"offset": 0, "length": 3, "string": "foo", "captures": []}, {"offset": 8, "length": 3, "string": "FOO", "captures": []}
2351 jq ´match("foo (?<bar123>bar)? foo"; "ig")´
2353 "foo bar foo foo foo"
2354 => {"offset": 0, "length": 11, "string": "foo bar foo", "captures": [{"offset": 4, "length": 3, "string": "bar", "name": "bar123"}]}, {"offset": 12, "length": 8, "string": "foo foo", "captures": [{"offset": -1, "length": 0, "string": null, "name": "bar123"}]}
2355 jq ´[ match("."; "g")] | length´
2357 "abc"
2358 => 3
2359 capture(val), capture(regex; flags)
2363 Collects the named captures in a JSON object, with the name of each
2364 capture as the key, and the matched string as the corresponding value.
2365 jq ´capture("(?<a>[a-z]+)-(?<n>[0-9]+)")´
2369 "xyzzy-14"
2370 => { "a": "xyzzy", "n": "14" }
2371 scan(regex), scan(regex; flags)
2375 Emit a stream of the non-overlapping substrings of the input that match
2376 the regex in accordance with the flags, if any have been specified. If
2377 there is no match, the stream is empty. To capture all the matches for
2378 each input string, use the idiom [ expr ], e.g. [ scan(regex) ].
2379 split(regex; flags)
2381 For backwards compatibility, split splits on a string, not a regex.
2382 splits(regex), splits(regex; flags)
2384 These provide the same results as their split counterparts, but as a
2385 stream instead of an array.
2386 sub(regex; tostring) sub(regex; string; flags)
2388 Emit the string obtained by replacing the first match of regex in the
2389 input string with tostring, after interpolation. tostring should be a
2390 jq string, and may contain references to named captures. The named cap‐
2391 tures are, in effect, presented as a JSON object (as constructed by
2392 capture) to tostring, so a reference to a captured variable named "x"
2393 would take the form: "(.x)".
2394 gsub(regex; string), gsub(regex; string; flags)
2396 gsub is like sub but all the non-overlapping occurrences of the regex
2397 are replaced by the string, after interpolation.
2398
2400 Variables are an absolute necessity in most programming languages, but
2401 they´re relegated to an "advanced feature" in jq.
2402 In most languages, variables are the only means of passing around data.
2404 If you calculate a value, and you want to use it more than once, you´ll
2405 need to store it in a variable. To pass a value to another part of the
2406 program, you´ll need that part of the program to define a variable (as
2407 a function parameter, object member, or whatever) in which to place the
2408 data.
2409 It is also possible to define functions in jq, although this is is a
2411 feature whose biggest use is defining jq´s standard library (many jq
2412 functions such as map and find are in fact written in jq).
2413 jq has reduction operators, which are very powerful but a bit tricky.
2415 Again, these are mostly used internally, to define some useful bits of
2416 jq´s standard library.
2417 It may not be obvious at first, but jq is all about generators (yes, as
2419 often found in other languages). Some utilities are provided to help
2420 deal with generators.
2421 Some minimal I/O support (besides reading JSON from standard input, and
2423 writing JSON to standard output) is available.
2424 Finally, there is a module/library system.
2426 Variable / Symbolic Binding Operator: ... as $identifier | ...
2428 In jq, all filters have an input and an output, so manual plumbing is
2429 not necessary to pass a value from one part of a program to the next.
2430 Many expressions, for instance a + b, pass their input to two distinct
2431 subexpressions (here a and b are both passed the same input), so vari‐
2432 ables aren´t usually necessary in order to use a value twice.
2433 For instance, calculating the average value of an array of numbers
2435 requires a few variables in most languages - at least one to hold the
2436 array, perhaps one for each element or for a loop counter. In jq, it´s
2437 simply add / length - the add expression is given the array and pro‐
2438 duces its sum, and the length expression is given the array and pro‐
2439 duces its length.
2440 So, there´s generally a cleaner way to solve most problems in jq than
2442 defining variables. Still, sometimes they do make things easier, so jq
2443 lets you define variables using expression as $variable. All variable
2444 names start with $. Here´s a slightly uglier version of the array-aver‐
2445 aging example:
2446 length as $array_length | add / $array_length
2450 We´ll need a more complicated problem to find a situation where using
2454 variables actually makes our lives easier.
2455 Suppose we have an array of blog posts, with "author" and "title"
2457 fields, and another object which is used to map author usernames to
2458 real names. Our input looks like:
2459 {"posts": [{"title": "Frist psot", "author": "anon"},
2463 {"title": "A well-written article", "author": "person1"}],
2464 "realnames": {"anon": "Anonymous Coward",
2465 "person1": "Person McPherson"}}
2466 We want to produce the posts with the author field containing a real
2470 name, as in:
2471 {"title": "Frist psot", "author": "Anonymous Coward"}
2475 {"title": "A well-written article", "author": "Person McPherson"}
2476 We use a variable, $names, to store the realnames object, so that we
2480 can refer to it later when looking up author usernames:
2481 .realnames as $names | .posts[] | {title, author: $names[.author]}
2485 The expression exp as $x | ... means: for each value of expression exp,
2489 run the rest of the pipeline with the entire original input, and with
2490 $x set to that value. Thus as functions as something of a foreach loop.
2491 Just as {foo} is a handy way of writing {foo: .foo}, so {$foo} is a
2493 handy way of writing {foo:$foo}.
2494 Multiple variables may be declared using a single as expression by pro‐
2496 viding a pattern that matches the structure of the input (this is known
2497 as "destructuring"):
2498 . as {realnames: $names, posts: [$first, $second]} | ...
2502 The variable declarations in array patterns (e.g., . as [$first, $sec‐
2506 ond]) bind to the elements of the array in from the element at index
2507 zero on up, in order. When there is no value at the index for an array
2508 pattern element, null is bound to that variable.
2509 Variables are scoped over the rest of the expression that defines them,
2511 so
2512 .realnames as $names | (.posts[] | {title, author: $names[.author]})
2516 will work, but
2520 (.realnames as $names | .posts[]) | {title, author: $names[.author]}
2524 won´t.
2528 For programming language theorists, it´s more accurate to say that jq
2530 variables are lexically-scoped bindings. In particular there´s no way
2531 to change the value of a binding; one can only setup a new binding with
2532 the same name, but which will not be visible where the old one was.
2533 jq ´.bar as $x | .foo | . + $x´
2537 {"foo":10, "bar":200}
2538 => 210
2539 jq ´. as $i|[(.*2|. as $i| $i), $i]´
2541 5
2542 => [10,5]
2543 jq ´. as [$a, $b, {c: $c}] | $a + $b + $c´
2545 [2, 3, {"c": 4, "d": 5}]
2546 => 9
2547 jq ´.[] as [$a, $b] | {a: $a, b: $b}´
2549 [[0], [0, 1], [2, 1, 0]]
2550 => {"a":0,"b":null}, {"a":0,"b":1}, {"a":2,"b":1}
2551 Defining Functions
2555 You can give a filter a name using "def" syntax:
2556 def increment: . + 1;
2560 From then on, increment is usable as a filter just like a builtin func‐
2564 tion (in fact, this is how many of the builtins are defined). A func‐
2565 tion may take arguments:
2566 def map(f): [.[] | f];
2570 Arguments are passed as filters (functions with no arguments), not as
2574 values. The same argument may be referenced multiple times with differ‐
2575 ent inputs (here f is run for each element of the input array). Argu‐
2576 ments to a function work more like callbacks than like value arguments.
2577 This is important to understand. Consider:
2578 def foo(f): f|f;
2582 5|foo(.*2)
2583 The result will be 20 because f is .*2, and during the first invocation
2587 of f . will be 5, and the second time it will be 10 (5 * 2), so the
2588 result will be 20. Function arguments are filters, and filters expect
2589 an input when invoked.
2590 If you want the value-argument behaviour for defining simple functions,
2592 you can just use a variable:
2593 def addvalue(f): f as $f | map(. + $f);
2597 Or use the short-hand:
2601 def addvalue($f): ...;
2605 With either definition, addvalue(.foo) will add the current input´s
2609 .foo field to each element of the array. Do note that calling
2610 addvalue(.[]) will cause the map(. + $f) part to be evaluated once per
2611 value in the value of . at the call site.
2612 Multiple definitions using the same function name are allowed. Each
2614 re-definition replaces the previous one for the same number of function
2615 arguments, but only for references from functions (or main program)
2616 subsequent to the re-definition. See also the section below on scoping.
2617 jq ´def addvalue(f): . + [f]; map(addvalue(.[0]))´
2621 [[1,2],[10,20]]
2622 => [[1,2,1], [10,20,10]]
2623 jq ´def addvalue(f): f as $x | map(. + $x); addvalue(.[0])´
2625 [[1,2],[10,20]]
2626 => [[1,2,1,2], [10,20,1,2]]
2627 Scoping
2631 There are two types of symbols in jq: value bindings (a.k.a., "vari‐
2632 ables"), and functions. Both are scoped lexically, with expressions
2633 being able to refer only to symbols that have been defined "to the
2634 left" of them. The only exception to this rule is that functions can
2635 refer to themselves so as to be able to create recursive functions.
2636 For example, in the following expression there is a binding which is
2638 visible "to the right" of it, ... | .*3 as $times_three | [. +
2639 $times_three] | ..., but not "to the left". Consider this expression
2640 now, ... | (.*3 as $times_three | [.+ $times_three]) | ...: here the
2641 binding $times_three is not visible past the closing parenthesis.
2642 Reduce
2644 The reduce syntax in jq allows you to combine all of the results of an
2645 expression by accumulating them into a single answer. As an example,
2646 we´ll pass [3,2,1] to this expression:
2647 reduce .[] as $item (0; . + $item)
2651 For each result that .[] produces, . + $item is run to accumulate a
2655 running total, starting from 0. In this example, .[] produces the
2656 results 3, 2, and 1, so the effect is similar to running something like
2657 this:
2658 0 | (3 as $item | . + $item) |
2662 (2 as $item | . + $item) |
2663 (1 as $item | . + $item)
2664 jq ´reduce .[] as $item (0; . + $item)´
2666 [10,2,5,3]
2667 => 20
2668 isempty(exp)
2672 Returns true if exp produces no outputs, false otherwise.
2673 jq ´isempty(empty)´
2677 null
2678 => true
2679 limit(n; exp)
2683 The limit function extracts up to n outputs from exp.
2684 jq ´[limit(3;.[])]´
2688 [0,1,2,3,4,5,6,7,8,9]
2689 => [0,1,2]
2690 first(expr), last(expr), nth(n; expr)
2694 The first(expr) and last(expr) functions extract the first and last
2695 values from expr, respectively.
2696 The nth(n; expr) function extracts the nth value output by expr. This
2698 can be defined as def nth(n; expr): last(limit(n + 1; expr));. Note
2699 that nth(n; expr) doesn´t support negative values of n.
2700 jq ´[first(range(.)), last(range(.)), nth(./2; range(.))]´
2704 10
2705 => [0,9,5]
2706 first, last, nth(n)
2710 The first and last functions extract the first and last values from any
2711 array at ..
2712 The nth(n) function extracts the nth value of any array at ..
2714 jq ´[range(.)]|[first, last, nth(5)]´
2718 10
2719 => [0,9,5]
2720 foreach
2724 The foreach syntax is similar to reduce, but intended to allow the con‐
2725 struction of limit and reducers that produce intermediate results (see
2726 example).
2727 The form is foreach EXP as $var (INIT; UPDATE; EXTRACT). Like reduce,
2729 INIT is evaluated once to produce a state value, then each output of
2730 EXP is bound to $var, UPDATE is evaluated for each output of EXP with
2731 the current state and with $var visible. Each value output by UPDATE
2732 replaces the previous state. Finally, EXTRACT is evaluated for each new
2733 state to extract an output of foreach.
2734 This is mostly useful only for constructing reduce- and limit-like
2736 functions. But it is much more general, as it allows for partial reduc‐
2737 tions (see the example below).
2738 jq ´[foreach .[] as $item ([[],[]]; if $item == null then [[],.[0]] else [(.[0] + [$item]),[]] end; if $item == null then .[1] else empty end)]´
2742 [1,2,3,4,null,"a","b",null]
2743 => [[1,2,3,4],["a","b"]]
2744 Recursion
2748 As described above, recurse uses recursion, and any jq function can be
2749 recursive. The while builtin is also implemented in terms of recursion.
2750 Tail calls are optimized whenever the expression to the left of the
2752 recursive call outputs its last value. In practice this means that the
2753 expression to the left of the recursive call should not produce more
2754 than one output for each input.
2755 For example:
2757 def recurse(f): def r: ., (f | select(. != null) | r); r;
2761 def while(cond; update):
2763 def _while:
2764 if cond then ., (update | _while) else empty end;
2765 _while;
2766 def repeat(exp):
2768 def _repeat:
2769 exp, _repeat;
2770 _repeat;
2771 Generators and iterators
2775 Some jq operators and functions are actually generators in that they
2776 can produce zero, one, or more values for each input, just as one might
2777 expect in other programming languages that have generators. For exam‐
2778 ple, .[] generates all the values in its input (which must be an array
2779 or an object), range(0; 10) generates the integers between 0 and 10,
2780 and so on.
2781 Even the comma operator is a generator, generating first the values
2783 generated by the expression to the left of the comma, then for each of
2784 those, the values generate by the expression on the right of the comma.
2785 The empty builtin is the generator that produces zero outputs. The
2787 empty builtin backtracks to the preceding generator expression.
2788 All jq functions can be generators just by using builtin generators. It
2790 is also possible to define new generators using only recursion and the
2791 comma operator. If the recursive call(s) is(are) "in tail position"
2792 then the generator will be efficient. In the example below the recur‐
2793 sive call by _range to itself is in tail position. The example shows
2794 off three advanced topics: tail recursion, generator construction, and
2795 sub-functions.
2796 jq ´def range(init; upto; by): def _range: if (by > 0 and . < upto) or (by < 0 and . > upto) then ., ((.+by)|_range) else . end; if by == 0 then init else init|_range end | select((by > 0 and . < upto) or (by < 0 and . > upto)); range(0; 10; 3)´
2800 null
2801 => 0, 3, 6, 9
2802 jq ´def while(cond; update): def _while: if cond then ., (update | _while) else empty end; _while; [while(.<100; .*2)]´
2804 1
2805 => [1,2,4,8,16,32,64]
2806
2810 jq currently only has IEEE754 double-precision (64-bit) floating point
2811 number support.
2812 Besides simple arithmetic operators such as +, jq also has most stan‐
2814 dard math functions from the C math library. C math functions that take
2815 a single input argument (e.g., sin()) are available as zero-argument jq
2816 functions. C math functions that take two input arguments (e.g., pow())
2817 are available as two-argument jq functions that ignore .. C math func‐
2818 tions that take three input arguments are available as three-argument
2819 jq functions that ignore ..
2820 Availability of standard math functions depends on the availability of
2822 the corresponding math functions in your operating system and C math
2823 library. Unavailable math functions will be defined but will raise an
2824 error.
2825 One-input C math functions: acos acosh asin asinh atan atanh cbrt ceil
2827 cos cosh erf erfc exp exp10 exp2 expm1 fabs floor gamma j0 j1 lgamma
2828 log log10 log1p log2 logb nearbyint pow10 rint round significand sin
2829 sinh sqrt tan tanh tgamma trunc y0 y1.
2830 Two-input C math functions: atan2 copysign drem fdim fmax fmin fmod
2832 frexp hypot jn ldexp modf nextafter nexttoward pow remainder scalb
2833 scalbln yn.
2834 Three-input C math functions: fma.
2836 See your system´s manual for more information on each of these.
2838
2840 At this time jq has minimal support for I/O, mostly in the form of con‐
2841 trol over when inputs are read. Two builtins functions are provided for
2842 this, input and inputs, that read from the same sources (e.g., stdin,
2843 files named on the command-line) as jq itself. These two builtins, and
2844 jq´s own reading actions, can be interleaved with each other.
2845 Two builtins provide minimal output capabilities, debug, and stderr.
2847 (Recall that a jq program´s output values are always output as JSON
2848 texts on stdout.) The debug builtin can have application-specific
2849 behavior, such as for executables that use the libjq C API but aren´t
2850 the jq executable itself. The stderr builtin outputs its input in raw
2851 mode to stder with no additional decoration, not even a newline.
2852 Most jq builtins are referentially transparent, and yield constant and
2854 repeatable value streams when applied to constant inputs. This is not
2855 true of I/O builtins.
2856 input
2858 Outputs one new input.
2859 inputs
2861 Outputs all remaining inputs, one by one.
2862 This is primarily useful for reductions over a program´s inputs.
2864 debug
2866 Causes a debug message based on the input value to be produced. The jq
2867 executable wraps the input value with ["DEBUG:", <input-value>] and
2868 prints that and a newline on stderr, compactly. This may change in the
2869 future.
2870 stderr
2872 Prints its input in raw and compact mode to stderr with no additional
2873 decoration, not even a newline.
2874 input_filename
2876 Returns the name of the file whose input is currently being filtered.
2877 Note that this will not work well unless jq is running in a UTF-8
2878 locale.
2879 input_line_number
2881 Returns the line number of the input currently being filtered.
2882
2884 With the --stream option jq can parse input texts in a streaming fash‐
2885 ion, allowing jq programs to start processing large JSON texts immedi‐
2886 ately rather than after the parse completes. If you have a single JSON
2887 text that is 1GB in size, streaming it will allow you to process it
2888 much more quickly.
2889 However, streaming isn´t easy to deal with as the jq program will have
2891 [<path>, <leaf-value>] (and a few other forms) as inputs.
2892 Several builtins are provided to make handling streams easier.
2894 The examples below use the streamed form of [0,[1]], which is
2896 [[0],0],[[1,0],1],[[1,0]],[[1]].
2897 Streaming forms include [<path>, <leaf-value>] (to indicate any scalar
2899 value, empty array, or empty object), and [<path>] (to indicate the end
2900 of an array or object). Future versions of jq run with --stream and
2901 -seq may output additional forms such as ["error message"] when an
2902 input text fails to parse.
2903 truncate_stream(stream_expression)
2905 Consumes a number as input and truncates the corresponding number of
2906 path elements from the left of the outputs of the given streaming
2907 expression.
2908 jq ´[1|truncate_stream([[0],1],[[1,0],2],[[1,0]],[[1]])]´
2912 1
2913 => [[[0],2],[[0]]]
2914 fromstream(stream_expression)
2918 Outputs values corresponding to the stream expression´s outputs.
2919 jq ´fromstream(1|truncate_stream([[0],1],[[1,0],2],[[1,0]],[[1]]))´
2923 null
2924 => [2]
2925 tostream
2929 The tostream builtin outputs the streamed form of its input.
2930 jq ´. as $dot|fromstream($dot|tostream)|.==$dot´
2934 [0,[1,{"a":1},{"b":2}]]
2935 => true
2936
2940 Assignment works a little differently in jq than in most programming
2941 languages. jq doesn´t distinguish between references to and copies of
2942 something - two objects or arrays are either equal or not equal, with‐
2943 out any further notion of being "the same object" or "not the same
2944 object".
2945 If an object has two fields which are arrays, .foo and .bar, and you
2947 append something to .foo, then .bar will not get bigger, even if you´ve
2948 previously set .bar = .foo. If you´re used to programming in languages
2949 like Python, Java, Ruby, Javascript, etc. then you can think of it as
2950 though jq does a full deep copy of every object before it does the
2951 assignment (for performance it doesn´t actually do that, but that´s the
2952 general idea).
2953 This means that it´s impossible to build circular values in jq (such as
2955 an array whose first element is itself). This is quite intentional, and
2956 ensures that anything a jq program can produce can be represented in
2957 JSON.
2958 All the assignment operators in jq have path expressions on the
2960 left-hand side (LHS). The right-hand side (RHS) procides values to set
2961 to the paths named by the LHS path expressions.
2962 Values in jq are always immutable. Internally, assignment works by
2964 using a reduction to compute new, replacement values for . that have
2965 had all the desired assignments applied to ., then outputting the modi‐
2966 fied value. This might be made clear by this example: {a:{b:{c:1}}} |
2967 (.a.b|=3), .. This will output {"a":{"b":3}} and {"a":{"b":{"c":1}}}
2968 because the last sub-expression, ., sees the original value, not the
2969 modified value.
2970 Most users will want to use modification assignment operators, such as
2972 |= or +=, rather than =.
2973 Note that the LHS of assignment operators refers to a value in .. Thus
2975 $var.foo = 1 won´t work as expected ($var.foo is not a valid or useful
2976 path expression in .); use $var | .foo = 1 instead.
2977 Note too that .a,.b=0 does not set .a and .b, but (.a,.b)=0 sets both.
2979 Update-assignment: |=
2981 This is the "update" operator ´|=´. It takes a filter on the right-hand
2982 side and works out the new value for the property of . being assigned
2983 to by running the old value through this expression. For instance,
2984 (.foo, .bar) |= .+1 will build an object with the "foo" field set to
2985 the input´s "foo" plus 1, and the "bar" field set to the input´s "bar"
2986 plus 1.
2987 The left-hand side can be any general path expression; see path().
2989 Note that the left-hand side of ´|=´ refers to a value in .. Thus
2991 $var.foo |= . + 1 won´t work as expected ($var.foo is not a valid or
2992 useful path expression in .); use $var | .foo |= . + 1 instead.
2993 If the right-hand side outputs no values (i.e., empty), then the
2995 left-hand side path will be deleted, as with del(path).
2996 If the right-hand side outputs multiple values, only the first one will
2998 be used (COMPATIBILITY NOTE: in jq 1.5 and earlier releases, it used to
2999 be that only the last one was used).
3000 jq ´(..|select(type=="boolean")) |= if . then 1 else 0 end´
3004 [true,false,[5,true,[true,[false]],false]]
3005 => [1,0,[5,1,[1,[0]],0]]
3006 Arithmetic update-assignment: +=, -=, *=, /=, %=, //=
3010 jq has a few operators of the form a op= b, which are all equivalent to
3011 a |= . op b. So, += 1 can be used to increment values, being the same
3012 as |= . + 1.
3013 jq ´.foo += 1´
3017 {"foo": 42}
3018 => {"foo": 43}
3019 Plain assignment: =
3023 This is the plain assignment operator. Unlike the others, the input to
3024 the right-hand-side (RHS) is the same as the input to the
3025 left-hand-side (LHS) rather than the value at the LHS path, and all
3026 values output by the RHS will be used (as shown below).
3027 If the RHS of ´=´ produces multiple values, then for each such value jq
3029 will set the paths on the left-hand side to the value and then it will
3030 output the modified .. For example, (.a,.b)=range(2) outputs
3031 {"a":0,"b":0}, then {"a":1,"b":1}. The "update" assignment forms (see
3032 above) do not do this.
3033 This example should show the difference between ´=´ and ´|=´:
3035 Provide input ´{"a": {"b": 10}, "b": 20}´ to the programs:
3037 .a = .b
3039 .a |= .b
3041 The former will set the "a" field of the input to the "b" field of the
3043 input, and produce the output {"a": 20, "b": 20}. The latter will set
3044 the "a" field of the input to the "a" field´s "b" field, producing
3045 {"a": 10, "b": 20}.
3046 Another example of the difference between ´=´ and ´|=´:
3048 null|(.a,.b)=range(3)
3050 outputs ´{"a":0,"b":0}´, ´{"a":1,"b":1}´, and ´{"a":2,"b":2}´, while
3052 null|(.a,.b)|=range(3)
3054 outputs just ´{"a":0,"b":0}´.
3056 Complex assignments
3058 Lots more things are allowed on the left-hand side of a jq assignment
3059 than in most languages. We´ve already seen simple field accesses on the
3060 left hand side, and it´s no surprise that array accesses work just as
3061 well:
3062 .posts[0].title = "JQ Manual"
3066 What may come as a surprise is that the expression on the left may pro‐
3070 duce multiple results, referring to different points in the input docu‐
3071 ment:
3072 .posts[].comments |= . + ["this is great"]
3076 That example appends the string "this is great" to the "comments" array
3080 of each post in the input (where the input is an object with a field
3081 "posts" which is an array of posts).
3082 When jq encounters an assignment like ´a = b´, it records the "path"
3084 taken to select a part of the input document while executing a. This
3085 path is then used to find which part of the input to change while exe‐
3086 cuting the assignment. Any filter may be used on the left-hand side of
3087 an equals - whichever paths it selects from the input will be where the
3088 assignment is performed.
3089 This is a very powerful operation. Suppose we wanted to add a comment
3091 to blog posts, using the same "blog" input above. This time, we only
3092 want to comment on the posts written by "stedolan". We can find those
3093 posts using the "select" function described earlier:
3094 .posts[] | select(.author == "stedolan")
3098 The paths provided by this operation point to each of the posts that
3102 "stedolan" wrote, and we can comment on each of them in the same way
3103 that we did before:
3104 (.posts[] | select(.author == "stedolan") | .comments) |=
3108 . + ["terrible."]
3109
3113 jq has a library/module system. Modules are files whose names end in
3114 .jq.
3115 Modules imported by a program are searched for in a default search path
3117 (see below). The import and include directives allow the importer to
3118 alter this path.
3119 Paths in the a search path are subject to various substitutions.
3121 For paths starting with "~/", the user´s home directory is substituted
3123 for "~".
3124 For paths starting with "$ORIGIN/", the path of the jq executable is
3126 substituted for "$ORIGIN".
3127 For paths starting with "./" or paths that are ".", the path of the
3129 including file is substituted for ".". For top-level programs given on
3130 the command-line, the current directory is used.
3131 Import directives can optionally specify a search path to which the
3133 default is appended.
3134 The default search path is the search path given to the -L command-line
3136 option, else ["~/.jq", "$ORIGIN/../lib/jq", "$ORIGIN/../lib"].
3137 Null and empty string path elements terminate search path processing.
3139 A dependency with relative path "foo/bar" would be searched for in
3141 "foo/bar.jq" and "foo/bar/bar.jq" in the given search path. This is
3142 intended to allow modules to be placed in a directory along with, for
3143 example, version control files, README files, and so on, but also to
3144 allow for single-file modules.
3145 Consecutive components with the same name are not allowed to avoid
3147 ambiguities (e.g., "foo/foo").
3148 For example, with -L$HOME/.jq a module foo can be found in
3150 $HOME/.jq/foo.jq and $HOME/.jq/foo/foo.jq.
3151 If "$HOME/.jq" is a file, it is sourced into the main program.
3153 import RelativePathString as NAME [<metadata>];
3155 Imports a module found at the given path relative to a directory in a
3156 search path. A ".jq" suffix will be added to the relative path string.
3157 The module´s symbols are prefixed with "NAME::".
3158 The optional metadata must be a constant jq expression. It should be an
3160 object with keys like "homepage" and so on. At this time jq only uses
3161 the "search" key/value of the metadata. The metadata is also made
3162 available to users via the modulemeta builtin.
3163 The "search" key in the metadata, if present, should have a string or
3165 array value (array of strings); this is the search path to be prefixed
3166 to the top-level search path.
3167 include RelativePathString [<metadata>];
3169 Imports a module found at the given path relative to a directory in a
3170 search path as if it were included in place. A ".jq" suffix will be
3171 added to the relative path string. The module´s symbols are imported
3172 into the caller´s namespace as if the module´s content had been
3173 included directly.
3174 The optional metadata must be a constant jq expression. It should be an
3176 object with keys like "homepage" and so on. At this time jq only uses
3177 the "search" key/value of the metadata. The metadata is also made
3178 available to users via the modulemeta builtin.
3179 import RelativePathString as $NAME [<metadata>];
3181 Imports a JSON file found at the given path relative to a directory in
3182 a search path. A ".json" suffix will be added to the relative path
3183 string. The file´s data will be available as $NAME::NAME.
3184 The optional metadata must be a constant jq expression. It should be an
3186 object with keys like "homepage" and so on. At this time jq only uses
3187 the "search" key/value of the metadata. The metadata is also made
3188 available to users via the modulemeta builtin.
3189 The "search" key in the metadata, if present, should have a string or
3191 array value (array of strings); this is the search path to be prefixed
3192 to the top-level search path.
3193 module <metadata>;
3195 This directive is entirely optional. It´s not required for proper oper‐
3196 ation. It serves only the purpose of providing metadata that can be
3197 read with the modulemeta builtin.
3198 The metadata must be a constant jq expression. It should be an object
3200 with keys like "homepage". At this time jq doesn´t use this metadata,
3201 but it is made available to users via the modulemeta builtin.
3202 modulemeta
3204 Takes a module name as input and outputs the module´s metadata as an
3205 object, with the module´s imports (including metadata) as an array
3206 value for the "deps" key.
3207 Programs can use this to query a module´s metadata, which they could
3209 then use to, for example, search for, download, and install missing
3210 dependencies.
3211
3213 To configure alternative colors just set the JQ_COLORS environment
3214 variable to colon-delimited list of partial terminal escape sequences
3215 like "1;31", in this order:
3216 · color for null
3218 · color for false
3220 · color for true
3222 · color for numbers
3224 · color for strings
3226 · color for arrays
3228 · color for objects
3230 The default color scheme is the same as setting "JQ_COL‐
3234 ORS=1;30:0;39:0;39:0;39:0;32:1;39:1;39".
3235 This is not a manual for VT100/ANSI escapes. However, each of these
3237 color specifications should consist of two numbers separated by a
3238 semi-colon, where the first number is one of these:
3239 · 1 (bright)
3241 · 2 (dim)
3243 · 4 (underscore)
3245 · 5 (blink)
3247 · 7 (reverse)
3249 · 8 (hidden)
3251 and the second is one of these:
3255 · 30 (black)
3257 · 31 (red)
3259 · 32 (green)
3261 · 33 (yellow)
3263 · 34 (blue)
3265 · 35 (magenta)
3267 · 36 (cyan)
3269 · 37 (white)
3271
3275 Presumably. Report them or discuss them at:
3276 https://github.com/stedolan/jq/issues
3280
3284 Stephen Dolan <mu@netsoc.tcd.ie>
3285 December 2017 JQ(1)